How to process multiple files parallely Post: 302717301

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to process multiple files in Korn Shell

How do I make the below ksh to process all of the files inside a user specified directory? Currently it can only process one file at a time. #!/bin/ksh tr -s '\11 ' ' ' < $1 > temp0 sed -e 's/,//g' temp0 > temp1 cut -d' ' -f1,4,5 temp1 > final_output rm temp0 temp1

2. UNIX for Dummies Questions & Answers

How do I Process Multiple Files

Hi, First post here. In Mac OSX terminal I need to run a program against multiple files in a directory and append the output to tab separated variable file. I currently type the following to process just one file MBP:/Users/dc1743/Desktop/SFT root# ./myprogram myfile1.plist >>...

3. Programming

Process multiple pcap files at once

Hi all, I'm writing a program using libpcap, and I have multiple pcap files in a folder that I want to capture. I currently have handle = pcap_open_offline("/data/traffic/pcap1.pcap", errbuf"); which works fine since pcap_open_offline() takes in a filename. However, I want to process...

4. Shell Programming and Scripting

executing 2 or more files inside another file parallely

hi All, i have 3 files file1 file2 file3 these 3 files are shell (KSH) scripts. i want to place all these 3 shell scripts in another file file99 and execute these files simultaneoulsy inside the file99 these 3 files should be present and executed simultaneously when i excute file99 (nohup...

5. Shell Programming and Scripting

Multiple Threads/Tasks to run parallely using the shell script

Scenario: I have two PCs (named as A & B) which would send some traps to my third PC (named as C). In PC C, I have to write a shell script such that it should accept the datas from both the PC-A & B parallely. So my question is, is it possible to have two different child threads/tasks...

6. Shell Programming and Scripting

How to process multiple input files using Shell scripting

Hi, I would like to write a for loop that does the following: I have a file called X.txt and other files called 1.txt,2.txt, .....,1000.txt. I want to substitute the 6th column of the file X.txt with 1.txt and store the output as X.1. Then I want to do the same with X.txt and 2.txt and store...

7. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk...

8. Shell Programming and Scripting

Run multiple procedures from shell script parallely

Hi, I need to write a Shell Script wherein i will connect to a DB2 Database and run multiple DB procedures. I know how to do in a way where procedures will be called one after the other, like when first procedure finishes, second will be executed. But what i want is to run them at the same time...

9. Shell Programming and Scripting

Process multiple large files with awk

Hi there, I'm camor and I'm trying to process huge files with bash scripting and awk. I've got a dataset folder with 10 files (16 millions of row each one - 600MB), and I've got a sorted file with all keys inside. For example: a sample_1 200 a.b sample_2 10 a sample_3 10 a sample_1 10 a...

10. Shell Programming and Scripting

Xargs to call python executable to process multiple bam files

I am running the below loop that to process the 3 bam files (which isn't always the case). A .py executable is then called using | xargs sh to further process. If I just run it with echo the output is fine and expected, however when | xargs sh is added I get the error. I tried adding | xargs...

LEARN ABOUT SUNOS

comm

comm(1) 							   User Commands							   comm(1)

NAME

       comm - select or reject lines common to two files

SYNOPSIS

       comm [-123] file1 file2

DESCRIPTION

       The comm utility reads file1 and file2, which must be ordered in the current collating sequence, and produces three text columns as output:
       lines only in file1; lines only in file2; and lines in both files.

       If the input files were ordered according to the collating sequence of the current locale, the lines  written  will  be	in  the  collating
       sequence of the original lines. If not, the results are unspecified.

OPTIONS

       The following options are supported:

       -1	Suppresses the output column of lines unique to file1.

       -2	Suppresses the output column of lines unique to file2.

       -3	Suppresses the output column of lines duplicated in file1 and file2.

OPERANDS

       The following operands are supported:

       file1	A path name of the first file to be compared. If file1 is -, the standard input is used.

       file2	A path name of the second file to be compared. If file2 is -, the standard input is used.

USAGE

       See largefile(5) for the description of the behavior of comm when encountering files greater than or equal to 2 Gbyte ( 2**31 bytes).

EXAMPLES

       Example 1: Printing a list of utilities specified by files

       If file1, file2, and file3 each contain a sorted list of utilities, the command

       example% comm -23 file1 file2  | comm -23 - file3

       prints a list of utilities in file1 not specified by either of the other files. The entry:

       example% comm -12 file1 file2 | comm -12 - file3

       prints a list of utilities specified by all three files. And the entry:

       example% comm -12  file2 file3 | comm -23 -file1

       prints a list of utilities specified by both file2 and file3, but not specified in file1.

ENVIRONMENT VARIABLES

       See  environ(5)	for  descriptions  of  the  following  environment  variables that affect the execution of comm: LANG, LC_ALL, LC_COLLATE,
       LC_CTYPE, LC_MESSAGES, and NLSPATH.

EXIT STATUS

       The following exit values are returned:

       0	All input files were successfully output as specified.

       >0	An error occurred.

ATTRIBUTES

       See attributes(5) for descriptions of the following attributes:

       +-----------------------------+-----------------------------+
       |      ATTRIBUTE TYPE	     |	    ATTRIBUTE VALUE	   |
       +-----------------------------+-----------------------------+
       |Availability		     |SUNWesu			   |
       +-----------------------------+-----------------------------+
       |CSI			     |enabled			   |
       +-----------------------------+-----------------------------+
       |Interface Stability	     |Standard			   |
       +-----------------------------+-----------------------------+

SEE ALSO

       cmp(1), diff(1), sort(1), uniq(1), attributes(5), environ(5), largefile(5), standards(5)

SunOS 5.10							    3 Mar 2004								   comm(1)