How to process multiple input files using Shell scripting

7 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Export Oracle multiple tables to multiple csv files using UNIX shell scripting

Hello All, just wanted to export multiple tables from oracle sql using unix shell script to csv file and the below code is exporting only the first table. Can you please suggest why? or any better idea? export FILE="/abc/autom/file/geo_JOB.csv" Export= `sqlplus -s dev01/password@dEV3...

2. Shell Programming and Scripting

Shell Scripting - Select multiple files from numbered list

I am trying to have the user select two files from a numbered list which will eventually be turned into a variable then combined. This is probably something simple and stupid that I am doing. clear echo "Please Select the Show interface status file" select FILE1 in *; echo "Please Select the...

3. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao

4. UNIX for Dummies Questions & Answers

Writing a loop to process multiple input files by a shell script

I have multiple input files that I want to manipulate using a shell script. The files are called 250.1 through 250.1000 but I only want the script to manipulate 250.300 through 250.1000. Before I was using the following script to manipulate the text files: for i in 250.*; do || awk...

5. Shell Programming and Scripting

How to Process input files from folder in shell script?

Hi, I want to process all input files available into folder (C:\ShellPrg\InputFile\) Input files are abc.CSV , XYZ.zip (zip of CSV file), PQR.gz (zip of CSV file). I want to check the extension of file, If its .zip/.gz then need to unzip the file as .CSV I want to parse line by line of...

6. Shell Programming and Scripting

Background Process Shell Scripting

I have a following program: echofunc() { filename=$1 echo "reading $filename" while read line do echo $line; sleep 6; done < $filename } split -5 new.dat ls x* > input.dat while read file do echofun $file & done < input.dat

7. Shell Programming and Scripting

How to process multiple files in Korn Shell

How do I make the below ksh to process all of the files inside a user specified directory? Currently it can only process one file at a time. #!/bin/ksh tr -s '\11 ' ' ' < $1 > temp0 sed -e 's/,//g' temp0 > temp1 cut -d' ' -f1,4,5 temp1 > final_output rm temp0 temp1

LEARN ABOUT DEBIAN

parallel-slurp

PARALLEL-SLURP(1)														 PARALLEL-SLURP(1)

NAME

       parallel-slurp - copy files from listed hosts

SYNOPSIS

       parallel-slurp [OPTIONS] -h hosts.txt -L destdir remote local

DESCRIPTION

       pssh provides a number of commands for executing against a group of computers, using SSH. It's most useful for operating on clusters of
       homogenously-configured hosts.

       parallel-slurp gathers specified files from hosts you listed.

OPTIONS

       -r --recursive
	   recusively copy directories (OPTIONAL)

       -L --localdir
	   output directory for remote file copies

       -h --hosts
	   hosts file (each line "host[:port] [user]")

       -l --user
	   username (OPTIONAL)

       -p --par
	   max number of parallel threads (OPTIONAL)

       -o --outdir
	   output directory for stdout files (OPTIONAL)

       -e --errdir
	   output directory for stderr files (OPTIONAL)

       -t --timeout
	   timeout (secs) (-1 = no timeout) per host (OPTIONAL)

       -O --options
	   SSH options (OPTIONAL)

       -v --verbose
	   turn on warning and diagnostic messages (OPTIONAL)

EXAMPLE

       An example to copy /home/irb2/foo.txt from each host. Files gathered will be stored in /tmp/outdir/hostname/foo.txt.

	   # prallel-slurp -h hosts.txt -L /tmp/outdir -l irb2 
		    /home/irb2/foo.txt foo.txt

ENVIRONMENT

       All four programs take similar sets of options. All of these options can be set using the following environment variables:

       o   PSSH_HOSTS
       o   PSSH_USER
       o   PSSH_PAR
       o   PSSH_OUTDIR
       o   PSSH_VERBOSE
       o   PSSH_OPTIONS

SEE ALSO

       parallel-ssh(1), parallel-scp(1), parallel-nuke(1), parallel-rsync(1), ssh(1)

AUTHOR

       Brent N. Chun <bnc@theether.org>

COPYING

       Copyright: 2003, 2004, 2005, 2006, 2007 Brent N. Chun

NOTES

	1. bnc@theether.org
	   mailto:bnc@theether.org

								    03/30/2009							 PARALLEL-SLURP(1)

Shell Programming and Scripting