I have a bash script that will take approx. 130 days to complete. I am trying to grep a list of 1,144 user ID's out of 41 (1 GB each) files. The 41 files were originally one 41 G file, but that was horrendously too slow.
This is my current file:
Currently, I am processing one of the 41 files every 4 minutes. 4 x 41 = 164 min.
164 / 60 (min/hour) = 2.73 hours per user_id. I have 1,144 user_id's multiplied by 2.73 = 3123.12 hours. 3123.12 / 24 (hours in a day) = 130.13 days.
As you can see, that is way too long to process this task. I don't know PERL but I've heard its faster. If anyone has any suggestions please let me know.
Last edited by vbe; 01-29-2010 at 10:54 AM..
Reason: code tags please
Hello experts,
I have a series issue in script that result with bad peformence and I wonder if you can assist me.
For example
I have two files:
File-New, size 15Mb.
File-Old, size 1Mb.
File-New content:
a
b
c
k
File-Old content:
d
f
a
b (0 Replies)
Hi,
I am running a BASH shell with the following script. The script works and gives me correct output but is very slow with large files. The more rows and columns (width and height) the slower as you can probably see.
How can I do what I want more efficiently? Any ideas welcome. It has been... (10 Replies)
I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file
STRUCTURE OF THE TWO FILES
The first file is a series of names
ramesh
sushil
jonga
sudesh
lugdi
whereas the second file (could be... (4 Replies)
My script builds a lot of these array lists, then compares their sizes which solves my problem, but runs very slow. :(
set -A comboSorted -- $(
for x in ${IDs}
do
nawk -v s=$x '
BEGIN { testPattern="^" s "$" }
{
if ( $2 ~ testPattern ) {
getline;getline;
if ($1 == "IMAGE_SIZE")
print... (1 Reply)
Basically my requirement is to know the total number of free anonymous ports.
anonymous port range is 32768- 65535.
i wrote a script for that
**********************************************
for i in {32768..65535}
do
netstat -an | grep $i > /dev/null
if
... (21 Replies)
Hi
I know the basic about script and sleep processes.
However this is more tricky:
I would like to run
sh -x /sbin/dhclient-script and slow down the output of the script as a whole.
How would you do it?
I would like to delay output on the screen with 1 second for every line for the output... (3 Replies)
I have shell program as below
#!/bin/sh
echo ======= LogManageri start ==========
#This directory is getting the raw data from remote server
Raw_data=/opt/ftplogs
# This directory is ready for process the data
Processing_dir=/opt/processing_dir
# This directory is prcoessed files and... (4 Replies)
Hello,
I am showing the start of my script.
I am finding that 'xmessage' is taking about 12-15 seconds to show.
This in a terminal is very quick '/opt/vc/bin/vcgencmd get_camera'.
Is there any way to get 'camera not detected' to show faster.
Regards
#!/bin/bash
s=$(/opt/vc/bin/vcgencmd... (4 Replies)
I'm new to utilities like socat and netcat and I'm not clear if they will do what I need.
I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes.
I'm using Cygwin bash on Windows 10.
My... (3 Replies)
Discussion started by: siegfried
3 Replies
LEARN ABOUT DEBIAN
pescetti
PESCETTI(1) General Commands Manual PESCETTI(1)NAME
pescetti -- Pseudo-Duplimate Generator
SYNOPSIS
pescetti
DESCRIPTION
This manual page documents briefly the pescetti command.
OPTIONS
Here are a list of the available options and what they do. You must specify exactly one from --demo, --generate or --load.
--help Prints the help text
--demo Demonstration mode. Generates one hand with permutations and the tutorial for how to use them.
--generate=N
Generate N random boards
--load=boards.txt
Load boards+analysis from boards.txt
--load-dds=boards.dds
Load boards from boards.dds in dds format
--load-analysis=tricks.txt
Load analysis from tricks.txt
--permutations=permutations.txt
Generate the permutations and save them to the given file
--curtains=curtains.txt
Save curtain cards to file curtains.txt
--save=boards.txt
Save the boards+analysis to boards.txt
--save-dds=boards.dds
Save the boards to boards.dds in dds format
--save-analysis=tricks.txt
Save the analysis to tricks.txt
--format=html|txt|pdf
Set the output mode to the given format
--title=title
Set the title for the output
--output=hands.txt
Print the hands to hands.txt, rather than to standard output
--stats Generate statistics about the set of boards; included in the hands output
--analyze Run the dds analyzer on the boards and print the resulting numberof tricks (warning SLOW)
--criteria=
A list of criteria to apply to each generated hand to generate specific hand types. The list should be space separated and each
item may be suffixed with a colon and a (fractional) probability value which can be used to weight the criteria.
E.g. --criteria="weaknt:0.8 strongnt:0.5"
Valid criteria are: unbalanced weaknt strongnt twont strongtwo weaktwo three twoclubs 4441 singlesuit twosuits partscore game
slam game-invite slam-invite jumpshift jumpfit splinter bacon weird
--probability=factor
Generate hands matching the criteria with only the given probability. Factor is in the range 0 to 1. On each attempt to generate
a board it is rejected if it doesn't match the criteria with the given probability. A factor of about 0.8 gives roughly half
matching boards
AUTHOR
This manual page was written by Matthew Johnson <debian@matthew.ath.cx>. Permission is granted to copy, distribute and/or modify this docu-
ment under the terms of the GNU General Public License, Version 2 as published by the Free Software Foundation.
On Debian systems, the complete text of the GNU General Public License can be found in /usr/share/common-licenses/GPL.
PESCETTI(1)