10-18-2012
How to process multiple files parallely
Hi,
I have a requirement to process multiple files in a directory parallely.Consider the below scenario:
In a directory there are three files file1,file2 and file3.When I use for loop each file will be executed in sequence but I want to process parallely.
Any Help would be appreciated.
Thanks,
Liyakath Ali.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
How do I make the below ksh to process all of the files inside a user specified directory?
Currently it can only process one file at a time.
#!/bin/ksh
tr -s '\11 ' ' ' < $1 > temp0
sed -e 's/,//g' temp0 > temp1
cut -d' ' -f1,4,5 temp1 > final_output
rm temp0 temp1 (3 Replies)
Discussion started by: stevefox
3 Replies
2. UNIX for Dummies Questions & Answers
Hi,
First post here.
In Mac OSX terminal I need to run a program against multiple files in a directory and append the output to tab separated variable file.
I currently type the following to process just one file
MBP:/Users/dc1743/Desktop/SFT root# ./myprogram myfile1.plist >>... (1 Reply)
Discussion started by: dc1743
1 Replies
3. Programming
Hi all,
I'm writing a program using libpcap, and I have multiple pcap files in a folder that I want to capture.
I currently have
handle = pcap_open_offline("/data/traffic/pcap1.pcap", errbuf");
which works fine since pcap_open_offline() takes in a filename. However, I want to process... (0 Replies)
Discussion started by: lancer6238
0 Replies
4. Shell Programming and Scripting
hi All,
i have 3 files
file1
file2
file3
these 3 files are shell (KSH) scripts. i want to place all these 3 shell scripts in another file file99 and execute these files simultaneoulsy
inside the file99 these 3 files should be present and executed simultaneously when i excute file99 (nohup... (1 Reply)
Discussion started by: dareman123
1 Replies
5. Shell Programming and Scripting
Scenario:
I have two PCs (named as A & B) which would send some traps to my third PC (named as C).
In PC C, I have to write a shell script such that it should accept the datas from both the PC-A & B parallely.
So my question is, is it possible to have two different child threads/tasks... (2 Replies)
Discussion started by: nthiruvenkatam
2 Replies
6. Shell Programming and Scripting
Hi,
I would like to write a for loop that does the following:
I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt.
I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store... (0 Replies)
Discussion started by: evelibertine
0 Replies
7. UNIX for Dummies Questions & Answers
I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files:
for i in 250.*; do
|| awk... (4 Replies)
Discussion started by: evelibertine
4 Replies
8. Shell Programming and Scripting
Hi,
I need to write a Shell Script wherein i will connect to a DB2 Database and run multiple DB procedures. I know how to do in a way where procedures will be called one after the other, like when first procedure finishes, second will be executed.
But what i want is to run them at the same time... (11 Replies)
Discussion started by: Neelkanth
11 Replies
9. Shell Programming and Scripting
Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk.
I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside.
For example:
a sample_1 200
a.b sample_2 10
a sample_3 10
a sample_1 10
a... (4 Replies)
Discussion started by: camor
4 Replies
10. Shell Programming and Scripting
I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when
| xargs sh is added I get the error. I tried adding | xargs... (4 Replies)
Discussion started by: cmccabe
4 Replies
DIFF3(1) General Commands Manual DIFF3(1)
NAME
diff3 - 3-way differential file comparison
SYNOPSIS
diff3 [ -exEX3 ] file1 file2 file3
DESCRIPTION
Diff3 compares three versions of a file, and publishes disagreeing ranges of text flagged with these codes:
==== all three files differ
====1 file1 is different
====2 file2 is different
====3 file3 is different
The type of change suffered in converting a given range of a given file to some other is indicated in one of these ways:
f : n1 a Text is to be appended after line number n1 in file f, where f = 1, 2, or 3.
f : n1 , n2 c Text is to be changed in the range line n1 to line n2. If n1 = n2, the range may be abbreviated to n1.
The original contents of the range follows immediately after a c indication. When the contents of two files are identical, the contents of
the lower-numbered file is suppressed.
Under the -e option, diff3 publishes a script for the editor ed that will incorporate into file1 all changes between file2 and file3, i.e.
the changes that normally would be flagged ==== and ====3. Option -x (-3) produces a script to incorporate only changes flagged ====
(====3). The following command will apply the resulting script to `file1'.
(cat script; echo '1,$p') | ed - file1
The -E and -X are similar to -e and -x, respectively, but treat overlapping changes (i.e., changes that would be flagged with ==== in the
normal listing) differently. The overlapping lines from both files will be inserted by the edit script, bracketed by "<<<<<<" and ">>>>>>"
lines.
For example, suppose lines 7-8 are changed in both file1 and file2. Applying the edit script generated by the command
"diff3 -E file1 file2 file3"
to file1 results in the file:
lines 1-6
of file1
<<<<<<< file1
lines 7-8
of file1
=======
lines 7-8
of file3
>>>>>>> file3
rest of file1
The -E option is used by RCS merge(1) to insure that overlapping changes in the merged files are preserved and brought to someone's atten-
tion.
FILES
/tmp/d3?????
/usr/libexec/diff3
SEE ALSO
diff(1)
BUGS
Text lines that consist of a single `.' will defeat -e.
7th Edition October 21, 1996 DIFF3(1)