perform 3 awk commands to multiple files in multiple directories Post: 302568464

Sponsored Content

Top Forums Shell Programming and Scripting perform 3 awk commands to multiple files in multiple directories Post 302568464 by amarn on Thursday 27th of October 2011 11:09:15 AM

10-27-2011

Registered User

Dear both thank you for your help , i've tried both your approaches, unfortunately none works

based on vgersh99

Code:

#!/usr/bin/sh

set -xv

find /home/tester/datasets_26_10_11/hour_1/ -name "cust_*.txt" -type f | \


while read fname
do
	fileBaseName=`basename "${fname}"`
	fileDirName=`dirname "${fname}"` 

	echo "fileBaseName: [${fileDirName}][${fileBaseName}] - fname[${fname}]"

#	awk -v outputPath="${fileDirName}" '{print $0 > outputPath "/" $2".txt"}' "${fname}"
#	awk -v outputPath="${fileDirName}" 'FNR>3 {print > outputPath "/"
"fin_"FILENAME}' "${fileDirName}/n_cust*"

	echo "now working on: [${fname}] with [${fileBaseName}]"
	
	
# testing only first awk command
	awk -v outputPath="${fileDirName}" 'BEGIN{RS =" ";}{print > outputPath "/" "n_"FILENAME}' "${fileBaseName}"


	echo "fileBasename AFTER FIRST AWK : [${fileBaseName}]"

#second awk command
#	awk -v outputPath="${fileDirName}" -v inputPath="${fileDirName}" 'FNR>3 {print > outputPath "/" "fin_"FILENAME}' ${fileBaseName}

done

based on CarloM

Code:

#!/usr/bin/sh

set -xv

find /home/tester/datasets_26_10_11/hour_1/ -name "cust_*.txt" -type f | \


while read fname
do
	fileBaseName=`basename "${fname}"`
	fileDirName=`dirname "${fname}"` 

	echo "fileBaseName: [${fileDirName}][${fileBaseName}] - fname[${fname}]"

#	awk -v outputPath="${fileDirName}" '{print $0 > outputPath "/" $2".txt"}' "${fname}"
#	awk -v outputPath="${fileDirName}" 'FNR>3 {print > outputPath "/" "fin_"FILENAME}' "${fileDirName}/n_cust*"

	echo "now working on: [${fname}] with [${fileBaseName}]"
	
	
	awk -v outputPath="${fileDirName}" -v inputPath="${fileDirName}" 'BEGIN{RS =" ";}{print > outputPath "/" "n_"FILENAME}' ${fileBaseName}


	echo "fileBasename AFTER FIRST AWK : [${fileBaseName}]"

#	awk -v outputPath="${fileDirName}" -v inputPath="${fileDirName}" 'FNR>3 {print > outputPath "/" "fin_"FILENAME}' ${fileBaseName}

done

and in both cases i get the awk error again

thanks again for the quick responses

amarn

View Public Profile for amarn

Find all posts by amarn

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Perform a command to multiple files

How do I perform a command to multiple files? For example, I want to look at all files in a directory and print the ones that do not contain a certain string. How do I go about doing this?

2. AIX

Script to perform some actions on multiple files

I have this Korn script that I wrote (with some help) that is run by cron. I basically watches a file system for a specific filename to be uploaded (via FTP), checks to make sure that the file is no longer being uploaded (by checking the files size), then runs a series of other scripts. The...

3. Shell Programming and Scripting

Multiple search string in multiple files using awk

Hi, filenames: contains name of list of files to search in. placelist contains the names of places to be searched in all files in "filenames" for i in $(<filenames) do egrep -f placelist $i if ] then echo $i fi done >> outputfile Output i am getting:

4. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Hello, I am trying to write a bash shell script that does the following: 1.Finds all *.txt files within my directory of interest 2. reads each of the files (25 files) one by one (tab-delimited format and have the same data format) 3. skips the first 10 rows of the file 4. extracts and...

5. UNIX for Dummies Questions & Answers

best method of replacing multiple strings in multiple files - sed or awk? most simple preferred :)

Hi guys, say I have a few files in a directory (58 text files or somthing) each one contains mulitple strings that I wish to replace with other strings so in these 58 files I'm looking for say the following strings: JAM (replace with BUTTER) BREAD (replace with CRACKER) SCOOP (replace...

6. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is...

7. Shell Programming and Scripting

FTP multiple files from multiple directories

I have multiple files that starts as TRADE_LOG spread across multiple folders in the given structure.. ./dir1/1/TRADE_LOG*.gz ./dir2/10/TRADE_LOG*.gz ./dir11/12/TRADE_LOG*.gz ./dir12/13/TRADE_LOG*.gz when I do ftp uisng mput from the "." dir I am getting the below given error mput...

8. UNIX for Dummies Questions & Answers

Deleting multiple directories inside multiple directories

Hi, Very unfamiliar with unix/linux stuff. Our admin is on vacation so, need help very quickly. I have directories (eg 40001, 40002, etc) that each have one subdirectory (01). Each subdir 01 has multiple subdirs (001, 002, 003, etc). They are same in each dir. I need to keep the top and...

9. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao

10. UNIX for Dummies Questions & Answers

Create multiple directories with awk

Hello all. Newbie here. In a directory, I have 50 files and one additional file that is a list of the names of the 50 files. I would like to create a directory for each of the 50 files, and I need the 50 directory names to correspond to the 50 file names. I know this can be done by running...

LEARN ABOUT CENTOS

zip_source_file

ZIP_SOURCE_FILE(3)					     Library Functions Manual						ZIP_SOURCE_FILE(3)

NAME

       zip_source_file - create data source from a file

LIBRARY

       libzip (-lzip)

SYNOPSIS

       struct zip_source * zip_source_file(struct zip *archive, const char *fname); "zip_uint64_t start" "zip_int64_t len"

DESCRIPTION

       The  function zip_source_file creates a zip source from a file.	zip_source_file opens fname and reads len bytes from offset start from it.
       If len is 0 or -1, the whole file (starting from start ) is used.

       The fie is opened and read when the data from the source is used, usually by zip_close.

RETURN VALUES

       Upon successful completion, the created source is returned.  Otherwise, NULL is returned and the error code in archive is set  to  indicate
       the error.

ERRORS

       zip_source_file fails if:

	      [ZIP_ER_INVAL]
		  fname, start, or len are invalid.

	      [ZIP_ER_MEMORY]
		  Required memory could not be allocated.

	      [ZIP_ER_OPEN]
		  Opening fname failed.

SEE ALSO

       libzip(3),    zip_add(3),    zip_replace(3),   zip_source_buffer(3),   zip_source_filep(3),   zip_source_free(3),   zip_source_function(3),
       zip_source_zip(3)

AUTHORS

       Dieter Baron <dillo@giga.or.at> and Thomas Klausner <tk@giga.or.at>

NiH								  August 1, 2008						ZIP_SOURCE_FILE(3)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Perform a command to multiple files

Discussion started by: mcgrawa

2. AIX

Script to perform some actions on multiple files

Discussion started by: heprox

3. Shell Programming and Scripting

Multiple search string in multiple files using awk

Discussion started by: pinnacle

4. Shell Programming and Scripting

extract multiple cloumns from multiple files; skip rows and include filenames; awk

Discussion started by: manishabh