![]() |
|
|
google unix.com
|
|||||||
| Forums | Register | Forum Rules | Links | Albums | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here. |
More UNIX and Linux Forum Topics You Might Find Helpful
|
||||
| Thread | Thread Starter | Forum | Replies | Last Post |
| Executing scripts in Parallel | itsme_maverick | Shell Programming and Scripting | 2 | 06-06-2008 12:23 PM |
| running servers parallel | soliberus | IP Networking | 1 | 06-03-2008 08:45 AM |
| Running scripts parallely | santho | UNIX for Dummies Questions & Answers | 4 | 07-23-2006 01:42 PM |
| Running three scripts parallelly | anwarsait | Shell Programming and Scripting | 1 | 07-18-2006 02:20 AM |
| running a parallel program | bankpro | High Level Programming | 4 | 01-20-2006 08:20 PM |
![]() |
|
|
LinkBack | Thread Tools | Search this Thread | Rate Thread | Display Modes |
|
|
|
||||
|
Running scripts in parallel
Hi, Iam having the scripts as follows. i jus want to run those in parallel. main aim is to minimise the time for overall execution of the script. now out.txt is having 1 lac records. script1(split.sh) Code:
split -1000 out.txt splitout
ls -A splitout* > filelist.txt
cat filelist.txt | while read fileln
do
parll.sh $fileln >> parll.nohup &
done
script2(parll.sh) Code:
cat $1 | while read line do nohup grep -h $line filename.txt >> nohup.out done main aim is to reduce the grep time from the filename.txt. so iam splitting the large file and passin gthe splitted file one by one and running the grep in parallel. Still it is taking more than 3.5 hours for lac records itself. iam having more than 5 lac records. so can anybody help me to reduce the time considerably. Thanks. |
|
||||
|
out.txt is a very large file to process
my motive is to quickly process the large file. so iam using spliting to split the large file and small files have datas(file1) and iam using that data's and greping that line from file2 and writing to an output file. To quick finish iam using nohup to run this spllited files paralley. suggesion pls?? |
|
||||
|
It would be my pleasure to solve your problem,
![]() However, just let me know the exact string, you want to search so that I can re-write it for faster processing. I just want this in simple grep syntax i.e.: - #grep "String" <filename> |
|
||||
|
out.txt file looks like QWERTYUI1234567890019289661 YUTTYYUI2763877779898588775 BVJHDFJFJ2678098802778897898 filename.txt will be Code:
QWERTYUI1234567890019289661 hgsfd@williams.com 287776666666778 2008-878-398 - 62638678 37872fwweiiilslvcxmnmn you can take the 2 scripts from my previous post. |
![]() |
| Bookmarks |
| Thread Tools | Search this Thread |
| Display Modes | Rate This Thread |
|
|