04-10-2007
concurrent processes
We have a very large text file..contains almost 100K lines.
We want to process this file to generate another text file as per our data requirement.
As for now the parsing of data takes 20-25 mins each , for 100K lines.
the current script uses :
While Read Each Line
do parsing..
Done
We want to cut down the runtime from 20-25 mins to aorund 5-10 mins..
Hence it was decided that we must split the large input file into 10 files of 10K lines each and run the parsing script for all the 10 files at the same time..something like concurrent run..
How do I achieve this..I am new to the concept of concurrent runs ...
Please guide.
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
OS - Sun OS7
What sources can I go to to figure out what is the maximun number of processes for OS7 with 2GB of memory.
I believe it is 64K processes, but this number reflects resources being swaped.
Any help is appreciated
SmartJuniorUnix (1 Reply)
Discussion started by: SmartJuniorUnix
1 Replies
2. UNIX for Dummies Questions & Answers
we've got solaris 5.6 installed in a ultra 5 box that serves as gateway server going to the main unix box.
just like to find out how to determine the number of concurrent terminal connections and processes that the ultra 5 box can handle? and handling at present time?
thanks in advance! (1 Reply)
Discussion started by: eddie_villarta
1 Replies
3. UNIX for Advanced & Expert Users
Hi,
I notice in my Sun Solaris 8 sparc workstatin, I am able to login concurrently using a same user ID.
Is there a way to disallow this? That is, at anyone time, the user can have only 1 login session.
How can it be done?
Thanks (10 Replies)
Discussion started by: champion
10 Replies
4. Shell Programming and Scripting
Hi
I have a ksh that can have multiple instances running at the same time.
The script (each instance) uses the SAME log file to write to.
Should this cause a problem or is the ksh clever enough to queue write requests to the file?
Thanks.
GMMIKE (2 Replies)
Discussion started by: GNMIKE
2 Replies
5. UNIX for Advanced & Expert Users
Is there a way to monitor certain processes and if they hang too long to kill them, but certain scripts which are expected to take a long time to let them go?
Thank you
Richard (4 Replies)
Discussion started by: ukndoit
4 Replies
6. Solaris
Hi
Is there an easy way to identify and group currently running processes into OS processes and APP processes. Not all applications are installed as packages.
Any free tools or scripts to do this?
Many thanks. (2 Replies)
Discussion started by: wilsonee
2 Replies
7. AIX
Hi,
on normal (non concurrent) vgs, it's possible to extend a lun on san-storage , and use chvg -g to rewrite vgda, and use disks with the new size for lvm operations
is the same procedure possbile on a hacmp-cluster (2 node in our case) with concurrent vgs in active/passive mode?
cheers... (5 Replies)
Discussion started by: funksen
5 Replies
8. Shell Programming and Scripting
Hi all,
I have a folder with sql files that need to be inserted in a DB with SQL*Plus. The thing is that it takes too long to insert them all one by one, so I want to insert them five at a time.
Currently what I use is this:
for $FILENAME in *.sql
do
sqlplus -s $DBUSER@$SID << EOF
... (0 Replies)
Discussion started by: Tr0cken
0 Replies
9. Shell Programming and Scripting
Hi,
The problem detail is follows
I have three individual scripts .
SCRIPT A
sh -x sqoop_channels_nc_daily_01.sh & sh -x sqoop_channels_nc_daily_02.sh & sh -x sqoop_channels_nc_daily_03.sh
SCRIPT B
sh -x sqoop_contacts_nc_daily_01.sh & sh -x sqoop_contacts_nc_daily_02.sh & sh -x... (1 Reply)
Discussion started by: H_bansal
1 Replies
10. AIX
Hi,
What are the differences between concurrent and enhanced concurrent VGs.?
Any advantages of enhanced concurrent VG over normal concurrent vg
Regards,
Siva (2 Replies)
Discussion started by: ksgnathan
2 Replies
CSREQ(1) BSD General Commands Manual CSREQ(1)
NAME
csreq -- Expert tool for manipulating Code Signing Requirement data
SYNOPSIS
csreq [-v] -r requirement-input -t
csreq [-v] -r requirement-input -b outputfile
DESCRIPTION
The csreq command manipulates Code Signing Requirement data. It reads one requirement from a file or command arguments, converts it into
internal form, checks it, and then optionally outputs it in a different form.
The options are as follows:
-b path
Requests that the requirement read be written in binary form to the path given.
-r requirement-input
Specifies the input requirement. See "specifying requirements" below. This is exactly the same format as is accepted by the -r and -R
options of the codesign(1) command.
-t Requests that the requirement read be written as text to standard output.
-v Increases the verbosity of output. Multiple instances of -v produce increasing levels of commentary output.
In the first synopsis form, csreq reads a Code Requirement and writes it to standard output as canonical source text. Note that with text
input, this actually compiles the requirement into internal form and then converts it back to text, giving you the system's view of the
requirement code.
In the second synopsis form, csreq reads a Code Requirement and writes its binary representation to a file. This is the same form produced by
the SecRequirementCopyData API, and is readily acceptable as input to Code Signing verification APIs. It can also be used as input to subse-
quent invocations of csreq by passing the filename to the -r option.
SPECIFYING REQUIREMENTS
The requirement argument (-r) can be given in various forms. A plain text argument is taken to be a path to a file containing the require-
ment. This program will accept both binary files containing properly compiled requirements code, and source files that are automatically com-
piled for use. An argument of "-" requests that the requirement(s) are read from standard input. Again, standard input can contain either
binary form or text. Finally, an argument that begins with an equal sign "=" is taken as a literal requirements source text, and is compiled
accordingly for use.
EXAMPLES
To compile an explicit requirement program and write its binary form to file "output":
csreq -r="identifier com.foo.test" -b output.csreq
To display the requirement program embedded at offset 1234 of file "foo":
tail -b 1234 foo | csreq -r- -t
FILES
DIAGNOSTICS
The csreq program exits 0 on success or 1 on failure. Errors in arguments yield exit code 2.
SEE ALSO
codesign(1)
HISTORY
The csreq command first appeared in Mac OS 10.5.0 .
BSD
June 1, 2006 BSD