Sponsored Content
Full Discussion: concurrent processes
Top Forums Shell Programming and Scripting concurrent processes Post 302113742 by Amruta Pitkar on Tuesday 10th of April 2007 12:07:08 AM
Old 04-10-2007
concurrent processes

We have a very large text file..contains almost 100K lines.
We want to process this file to generate another text file as per our data requirement.
As for now the parsing of data takes 20-25 mins each , for 100K lines.

the current script uses :
While Read Each Line
do parsing..
Done

We want to cut down the runtime from 20-25 mins to aorund 5-10 mins..
Hence it was decided that we must split the large input file into 10 files of 10K lines each and run the parsing script for all the 10 files at the same time..something like concurrent run..

How do I achieve this..I am new to the concept of concurrent runs ...
Please guide.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Max number of concurrent processes

OS - Sun OS7 What sources can I go to to figure out what is the maximun number of processes for OS7 with 2GB of memory. I believe it is 64K processes, but this number reflects resources being swaped. Any help is appreciated SmartJuniorUnix (1 Reply)
Discussion started by: SmartJuniorUnix
1 Replies

2. UNIX for Dummies Questions & Answers

concurrent terminal connections and processes

we've got solaris 5.6 installed in a ultra 5 box that serves as gateway server going to the main unix box. just like to find out how to determine the number of concurrent terminal connections and processes that the ultra 5 box can handle? and handling at present time? thanks in advance! (1 Reply)
Discussion started by: eddie_villarta
1 Replies

3. UNIX for Advanced & Expert Users

No concurrent login

Hi, I notice in my Sun Solaris 8 sparc workstatin, I am able to login concurrently using a same user ID. Is there a way to disallow this? That is, at anyone time, the user can have only 1 login session. How can it be done? Thanks (10 Replies)
Discussion started by: champion
10 Replies

4. Shell Programming and Scripting

Concurrent writing to file

Hi I have a ksh that can have multiple instances running at the same time. The script (each instance) uses the SAME log file to write to. Should this cause a problem or is the ksh clever enough to queue write requests to the file? Thanks. GMMIKE (2 Replies)
Discussion started by: GNMIKE
2 Replies

5. UNIX for Advanced & Expert Users

Monitoring Processes - Killing hung processes

Is there a way to monitor certain processes and if they hang too long to kill them, but certain scripts which are expected to take a long time to let them go? Thank you Richard (4 Replies)
Discussion started by: ukndoit
4 Replies

6. Solaris

Identifying and grouping OS processes and APP processes

Hi Is there an easy way to identify and group currently running processes into OS processes and APP processes. Not all applications are installed as packages. Any free tools or scripts to do this? Many thanks. (2 Replies)
Discussion started by: wilsonee
2 Replies

7. AIX

chvg -g on Concurrent VG

Hi, on normal (non concurrent) vgs, it's possible to extend a lun on san-storage , and use chvg -g to rewrite vgda, and use disks with the new size for lvm operations is the same procedure possbile on a hacmp-cluster (2 node in our case) with concurrent vgs in active/passive mode? cheers... (5 Replies)
Discussion started by: funksen
5 Replies

8. Shell Programming and Scripting

Concurrent execution

Hi all, I have a folder with sql files that need to be inserted in a DB with SQL*Plus. The thing is that it takes too long to insert them all one by one, so I want to insert them five at a time. Currently what I use is this: for $FILENAME in *.sql do sqlplus -s $DBUSER@$SID << EOF ... (0 Replies)
Discussion started by: Tr0cken
0 Replies

9. Shell Programming and Scripting

Help with how to run multiple script with concurrent processes runs sequentially.

Hi, The problem detail is follows I have three individual scripts . SCRIPT A sh -x sqoop_channels_nc_daily_01.sh & sh -x sqoop_channels_nc_daily_02.sh & sh -x sqoop_channels_nc_daily_03.sh SCRIPT B sh -x sqoop_contacts_nc_daily_01.sh & sh -x sqoop_contacts_nc_daily_02.sh & sh -x... (1 Reply)
Discussion started by: H_bansal
1 Replies

10. AIX

Difference between concurrent and enhanced concurrent VG

Hi, What are the differences between concurrent and enhanced concurrent VGs.? Any advantages of enhanced concurrent VG over normal concurrent vg Regards, Siva (2 Replies)
Discussion started by: ksgnathan
2 Replies
diff3(1)						      General Commands Manual							  diff3(1)

NAME
diff3 - 3-way differential file comparison SYNOPSIS
file1 file2 file3 DESCRIPTION
compares three versions of a file, and prints disagreeing ranges of text flagged with these codes: all three files differ file1 is different file2 is different file3 is different The type of change required to convert a given range of a given file to some other is indicated in one of these ways: Text is to be appended after line number n1 in file f, where f = or Text is to be changed in the range line n1 through line n2. If n1 = n2, the range can be abbreviated to n1. The original contents of the range follows immediately after a indication. When the contents of two files are identical, the contents of the lower-numbered file is suppressed. Produces a script for the editor that can be used to incorporate into file1 all changes between file2 and file3 (see ed(1)); i.e., the changes that normally would be flagged and Produces a script to incorporate only changes flagged Produces a script to incorporate only changes flagged Produces a script that will incorporate all changes between file2 and file3, but treat overlapping changes (that is, changes that would be flagged with in normal listing) differently. The overlapping lines in both files will be inserted by the edit script bracketed by and lines. Produces a script that will incorporate only changes flagged , but treat these changes in the manner of option. The following command applies the resulting script to file1. EXTERNAL INFLUENCES
International Code Set Support Single- and multi-byte character code sets are supported. WARNINGS
Text lines that consist of a single period defeat Files longer than 64K bytes do not work. FILES
SEE ALSO
diff(1). diff3(1)
All times are GMT -4. The time now is 06:12 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy