I have a bash script that will take approx. 130 days to complete. I am trying to grep a list of 1,144 user ID's out of 41 (1 GB each) files. The 41 files were originally one 41 G file, but that was horrendously too slow.
This is my current file:
Code:
#!/bin/bash
for i in `cat WashFD.txt` # 1,144 files
do
for b in `cat xfiles` # 41 "x??" files
do
echo "looking for " $i "in " $b
cat $b | grep -i $i >> SEID.searches
done
done
Currently, I am processing one of the 41 files every 4 minutes. 4 x 41 = 164 min.
164 / 60 (min/hour) = 2.73 hours per user_id. I have 1,144 user_id's multiplied by 2.73 = 3123.12 hours. 3123.12 / 24 (hours in a day) = 130.13 days.
As you can see, that is way too long to process this task. I don't know PERL but I've heard its faster. If anyone has any suggestions please let me know.
Last edited by vbe; 01-29-2010 at 10:54 AM..
Reason: code tags please
Hello experts,
I have a series issue in script that result with bad peformence and I wonder if you can assist me.
For example
I have two files:
File-New, size 15Mb.
File-Old, size 1Mb.
File-New content:
a
b
c
k
File-Old content:
d
f
a
b (0 Replies)
Hi,
I am running a BASH shell with the following script. The script works and gives me correct output but is very slow with large files. The more rows and columns (width and height) the slower as you can probably see.
How can I do what I want more efficiently? Any ideas welcome. It has been... (10 Replies)
I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file
STRUCTURE OF THE TWO FILES
The first file is a series of names
ramesh
sushil
jonga
sudesh
lugdi
whereas the second file (could be... (4 Replies)
My script builds a lot of these array lists, then compares their sizes which solves my problem, but runs very slow. :(
set -A comboSorted -- $(
for x in ${IDs}
do
nawk -v s=$x '
BEGIN { testPattern="^" s "$" }
{
if ( $2 ~ testPattern ) {
getline;getline;
if ($1 == "IMAGE_SIZE")
print... (1 Reply)
Basically my requirement is to know the total number of free anonymous ports.
anonymous port range is 32768- 65535.
i wrote a script for that
**********************************************
for i in {32768..65535}
do
netstat -an | grep $i > /dev/null
if
... (21 Replies)
Hi
I know the basic about script and sleep processes.
However this is more tricky:
I would like to run
sh -x /sbin/dhclient-script and slow down the output of the script as a whole.
How would you do it?
I would like to delay output on the screen with 1 second for every line for the output... (3 Replies)
I have shell program as below
#!/bin/sh
echo ======= LogManageri start ==========
#This directory is getting the raw data from remote server
Raw_data=/opt/ftplogs
# This directory is ready for process the data
Processing_dir=/opt/processing_dir
# This directory is prcoessed files and... (4 Replies)
Hello,
I am showing the start of my script.
I am finding that 'xmessage' is taking about 12-15 seconds to show.
This in a terminal is very quick '/opt/vc/bin/vcgencmd get_camera'.
Is there any way to get 'camera not detected' to show faster.
Regards
#!/bin/bash
s=$(/opt/vc/bin/vcgencmd... (4 Replies)
I'm new to utilities like socat and netcat and I'm not clear if they will do what I need.
I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes.
I'm using Cygwin bash on Windows 10.
My... (3 Replies)
Discussion started by: siegfried
3 Replies
LEARN ABOUT DEBIAN
net::dns::sec::tools::timetrans
timetrans(3pm) User Contributed Perl Documentation timetrans(3pm)NAME
Net::DNS::SEC::Tools::timetrans - Convert an integer seconds count into text units.
SYNOPSIS
use Net::DNS::SEC::Tools::timetrans;
$timestring = timetrans(86488);
$timestring = fuzzytimetrans(86488);
DESCRIPTION
The timetrans() interface in Net::DNS::SEC::Tools::timetrans converts an integer seconds count into the equivalent number of days, hours,
and minutes. The time converted is a relative time, not an absolute time. The returned time is given in terms of days, hours, minutes,
and seconds, as required to express the seconds count appropriately.
The fuzzytimetrans() interface converts an integer seconds count into the equivalent number of weeks or days or hours or minutes. The unit
chosen is that which is most natural for the seconds count. One decimal place of precision is included in the result.
INTERFACES
The interfaces to the Net::DNS::SEC::Tools::timetrans module are given below.
timetrans()
This routine converts an integer seconds count into the equivalent number of days, hours, and minutes. This converted seconds count is
returned as a text string. The seconds count must be greater than zero or an error will be returned.
Return Values:
If a valid seconds count was given, the count converted into the
appropriate text string will be returned.
An empty string is returned if no seconds count was given or if
the seconds count is less than one.
fuzzytimetrans()
This routine converts an integer seconds count into the equivalent number of weeks, days, hours, or minutes. This converted seconds count
is returned as a text string. The seconds count must be greater than zero or an error will be returned.
Return Values:
If a valid seconds count was given, the count converted into the
appropriate text string will be returned.
An empty string is returned if no seconds count was given or if
the seconds count is less than one.
EXAMPLES timetrans(400) returns 6 minutes, 40 seconds
timetrans(420) returns 7 minutes
timetrans(888) returns 14 minutes, 48 seconds
timetrans(86400) returns 1 day
timetrans(86488) returns 1 day, 28 seconds
timetrans(715000) returns 8 days, 6 hours, 36 minutes, 40 second
timetrans(720000) returns 8 days, 8 hours
fuzzytimetrans(400) returns 6.7 minutes
fuzzytimetrans(420) returns 7.0 minutes
fuzzytimetrans(888) returns 14.8 minutes
fuzzytimetrans(86400) returns 1.0 day
fuzzytimetrans(86488) returns 1.0 day
fuzzytimetrans(715000) returns 1.2 weeks
fuzzytimetrans(720000) returns 1.2 weeks
COPYRIGHT
Copyright 2004-2012 SPARTA, Inc. All rights reserved. See the COPYING file included with the DNSSEC-Tools package for details.
AUTHOR
Wayne Morrison, tewok@tislabs.com
SEE ALSO timetrans(1)perl v5.14.2 2012-06-18 timetrans(3pm)
01-29-2010
