multiple file processing


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers multiple file processing
# 1  
Old 04-30-2003
multiple file processing

I need to process a directory which will have a different amount of files in it from time to time. This is an error directory, I need to process each file indvidually in to one log file, then display the file to the user. So I would like to display the file name as well in the log file.
example

------------------------------------------------------------------------
File XXXXXX located in XXXXXXXXX
file details, blah blah blah.

------------------------------------------------------------------------
File XXXXXX located in XXXXXXXXX
file details, blah blah blah.

------------------------------------------------------------------------

is this possible?

Thanks for all the help.

Smilie
# 2  
Old 05-01-2003
A few posts like this one have addressed this before..

You can quickly concatenate all the files with this command if the xargs program is available:
Code:
cd /your/dir
(ls | xargs cat) > bigFile 2> /dev/null

Otherwise, you can use this (this will also allow you to add other text into bigFile), but it's pretty inefficient..
Code:
#! /usr/bin/ksh
cd /your/dir
for fn in `ls`
do
   echo "File $fn located in $PWD" >> bigFile
   cat bigFile $fn > TMP_00
   mv TMP_00 bigFile
done

# 3  
Old 05-02-2003
the cat statement seems to make absolutely no sense.

you can't output two files at the same time with cat, can you??
# 4  
Old 05-02-2003
Which block of code are you referring to?

In the first one, we list all the files and the xargs program tells the shell to concatenate everything into one big file. Any messages that would display (such as cat: input/output files 'bigFile' identical if bigFile is in the same directory as the files being concatenated) are sent to /dev/null so you don't see anything on screen..

In the second one, cat takes two files -- bigFile and $fn -- and concatenates them together into TMP_00.. remember, that's the whole purpose of "cat"!

Last edited by oombera; 02-18-2004 at 10:35 PM..
# 5  
Old 05-03-2003
guess I was wrong. i thought there was no way this code "cat bigFile $fn > TMP_00" would work
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Processing multiple files

Hello I have a program cfxfrwb which is designed to remove headers from reports files. The cfxfrwb is located in the following directory /u01/efin/v40/live/bin I run the program against a single report file in the temp directory and it does it's job../cfxfrwb... (2 Replies)
Discussion started by: captainrhodes
2 Replies

2. Shell Programming and Scripting

Add rules for multiple lines processing

Hi , I want to process the below file and add rules for grouping based on key in unix shell script. For eg Input file : I have 3 columns and key column is group Group status application G1 Complete A1 G1 Delay A2 G2 Complete A3,A4 G3 Delay A5, A6... (8 Replies)
Discussion started by: ashishagg2005
8 Replies

3. Shell Programming and Scripting

Plink (processing multiple commands) using Bash

I'm completely brand new to bash scripting (migrating from Windows batch file scripting). I'm currently trying to write a bash script that will automatically reset "error-disabled" Cisco switch ports. Please forgive the very crude and inefficient script I have so far (shown below). It is... (10 Replies)
Discussion started by: MKANET
10 Replies

4. Programming

awk processing / Shell Script Processing to remove columns text file

Hello, I extracted a list of files in a directory with the command ls . However this is not my computer, so the ls functionality has been revamped so that it gives the filesizes in front like this : This is the output of ls command : I stored the output in a file filelist 1.1M... (5 Replies)
Discussion started by: ajayram
5 Replies

5. Shell Programming and Scripting

Processing multiple files awk

hai i need my single awk script to act on 4 trace files of ns2 and to calculate througput and it should print result from each trace file in a single trace file. i tried with the following code but it doesnt work awk -f awkscript inputfile1 inputfile2 inputfile3 inputfile4>outputfile ... (4 Replies)
Discussion started by: sarathyy
4 Replies

6. Shell Programming and Scripting

multiple groups of files processing

I have five directories, dir1 to dir5 for each directory, I have all same number-named folders. There are four types of folders, {1..10}, {20..30}, { 40..50}, {60..70} Now for each types of folder, I will do the same thing, here is the code for i in {1..5} do cd dir$i mkdir temp1 for... (5 Replies)
Discussion started by: ksgreen
5 Replies

7. Shell Programming and Scripting

shell scripting-processing multiple scripts

I have four scripts to run. My 1st script will make script2 and script3 to run. I am setting a cron job for this script1 to run continuously. This script1 will check for 2 text files and based on the existance of those text files it will initiate the script2 and script3. Now my doubt is that... (2 Replies)
Discussion started by: RSC1985
2 Replies

8. UNIX for Dummies Questions & Answers

Script Problem-Processing multiple scripts

Hi All, I have four scripts to run. My 1st script will make script2 and script3 to run. I am setting a cron job for this script1 to run continuously. This script1 will check for 2 text files and based on the existance of those text files it will initiate the script2 and script3. Now my doubt... (2 Replies)
Discussion started by: RSC1985
2 Replies

9. UNIX for Dummies Questions & Answers

Multiple excel files processing on unix

Hi all, I am faced with a rather unusual problem regarding interaction between NT and UNIX. I am using an ETL (Extract-Transform-Load) tool on unix that has the capability to read .xls files. So, when I FTP an excel (.xls) file from a windows server to unix and attempt to read it with this... (3 Replies)
Discussion started by: ucode_2482
3 Replies

10. Shell Programming and Scripting

Processing Multiple Files

Hello Everyone, I am new to scripting and confused with how to do this efficiently. I am trying to use AWK to do this. I have a lot of files in a folder which has the data of my throughput measurements in two columns i.e. Serial # and Throughput. like this 177.994 847.9 178.996 ... (1 Reply)
Discussion started by: hakim
1 Replies
Login or Register to Ask a Question