I have a bash script that will take approx. 130 days to complete. I am trying to grep a list of 1,144 user ID's out of 41 (1 GB each) files. The 41 files were originally one 41 G file, but that was horrendously too slow.
This is my current file:
Code:
#!/bin/bash
for i in `cat WashFD.txt` # 1,144 files
do
for b in `cat xfiles` # 41 "x??" files
do
echo "looking for " $i "in " $b
cat $b | grep -i $i >> SEID.searches
done
done
Currently, I am processing one of the 41 files every 4 minutes. 4 x 41 = 164 min.
164 / 60 (min/hour) = 2.73 hours per user_id. I have 1,144 user_id's multiplied by 2.73 = 3123.12 hours. 3123.12 / 24 (hours in a day) = 130.13 days.
As you can see, that is way too long to process this task. I don't know PERL but I've heard its faster. If anyone has any suggestions please let me know.
Last edited by vbe; 01-29-2010 at 10:54 AM..
Reason: code tags please
Hello experts,
I have a series issue in script that result with bad peformence and I wonder if you can assist me.
For example
I have two files:
File-New, size 15Mb.
File-Old, size 1Mb.
File-New content:
a
b
c
k
File-Old content:
d
f
a
b (0 Replies)
Hi,
I am running a BASH shell with the following script. The script works and gives me correct output but is very slow with large files. The more rows and columns (width and height) the slower as you can probably see.
How can I do what I want more efficiently? Any ideas welcome. It has been... (10 Replies)
I had written a perl script to compare two files: new and master and get the output of the first file i.e. the first file: words that are not in the master file
STRUCTURE OF THE TWO FILES
The first file is a series of names
ramesh
sushil
jonga
sudesh
lugdi
whereas the second file (could be... (4 Replies)
My script builds a lot of these array lists, then compares their sizes which solves my problem, but runs very slow. :(
set -A comboSorted -- $(
for x in ${IDs}
do
nawk -v s=$x '
BEGIN { testPattern="^" s "$" }
{
if ( $2 ~ testPattern ) {
getline;getline;
if ($1 == "IMAGE_SIZE")
print... (1 Reply)
Basically my requirement is to know the total number of free anonymous ports.
anonymous port range is 32768- 65535.
i wrote a script for that
**********************************************
for i in {32768..65535}
do
netstat -an | grep $i > /dev/null
if
... (21 Replies)
Hi
I know the basic about script and sleep processes.
However this is more tricky:
I would like to run
sh -x /sbin/dhclient-script and slow down the output of the script as a whole.
How would you do it?
I would like to delay output on the screen with 1 second for every line for the output... (3 Replies)
I have shell program as below
#!/bin/sh
echo ======= LogManageri start ==========
#This directory is getting the raw data from remote server
Raw_data=/opt/ftplogs
# This directory is ready for process the data
Processing_dir=/opt/processing_dir
# This directory is prcoessed files and... (4 Replies)
Hello,
I am showing the start of my script.
I am finding that 'xmessage' is taking about 12-15 seconds to show.
This in a terminal is very quick '/opt/vc/bin/vcgencmd get_camera'.
Is there any way to get 'camera not detected' to show faster.
Regards
#!/bin/bash
s=$(/opt/vc/bin/vcgencmd... (4 Replies)
I'm new to utilities like socat and netcat and I'm not clear if they will do what I need.
I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes.
I'm using Cygwin bash on Windows 10.
My... (3 Replies)
Discussion started by: siegfried
3 Replies
LEARN ABOUT SUSE
date::manip::lang
Date::Manip::Lang(3) User Contributed Perl Documentation Date::Manip::Lang(3)NAME
Date::Manip::Lang - date manipulation routines (language initialization)
DESCRIPTION
This module is a series of routines, one per language, used to initialize the support for different languages in Date::Manip
ADDING A LANGUAGE
Adding a language is easily done. If you want to add a language, refer to the list of words and phrases given below. Translate them into
the desired language and email them to me.
Note that Date::Manip does support international character sets, so if there are non-ASCII characters in the words, it's not a problem. Be
sure to include an ASCII representation as well that can be used in cases where non-ASCII characters might cause problems. In many cases,
alternate spellings are allowed, and there may be multiple words or phrases which fit, so please include all of them (with ASCII
representations for any that include non-ASCII characters).
Please translate ALL of the following. In some cases, a phrase is given in parentheses. It is not necessary to translate the phrase.
They're there to show the word in the correct context.
month names (January February ...)
abbreviations (Jan Feb ...)
day name (Monday Tuesday ... Sunday)
abbreviation (Mon Tue ... Sun)
short abbrev. (M T ... S)
number suffix (1st 2nd ... 31st)
spelled out (first second ... thirty-first
now
today
tomorrow
yesterday
last (last day of the month)
each (each Tuesday of the month)
of (first day of the week)
at (at 3:00)
on (on Tuesday)
next (next Tuesday)
last (last Tuesday)
exactly (in exactly 3 hours)
approximately (in approximately 3 hours)
business (in 4 business days)
Some times of the day are named. At the very least, there is
probably noon and midnight. Provide all named times, and the
time of day.
noon 12:00:00
midnight 00:00:00
The delta field names can be written or abbreviated in many differet
ways. Provide all names and abbreviations for the seven fields. For
example:
years/year/yrs/yr/y
months/month/mon
weeks/week/wk/wks/w
days/day/d
hours/hour/hr/hrs
minutes/minute/min/mn
seconds/second/sec/s
What words/phrases can be used to say that a time is in the future? E.g.
IN 3 hours
3 hours LATER
3 hours IN THE FUTURE
In the past?
3 hours AGO
3 hours PAST
Does the language have an equivalent of the English AM/PM? If so,
what are all possible values of each?
Other than a comma or period, are there any common integer/decimal
separators? For example: 1.25 can be expressed as 1.25 or 1,25
commonly. Are there any other ways?
When expressing time the hours/minutes and minutes/seconds are
typically separated by colons. Are there any other separators?
If so, what combinations of the separators are used in real life?
For example: 05h30:00.
NOTE: there must be the same number of sephm and sepms values
and the first sephm corresponds to the first sepms, etc.
AUTHOR
Sullivan Beck (sbeck@cpan.org)
You can always get the newest beta version of Date::Manip (which may fix problems in the current CPAN version... and may add others) from
my home page:
http://www.cise.ufl.edu/~sbeck/
perl v5.12.1 2010-01-12 Date::Manip::Lang(3)
01-29-2010
