Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Iterating awk over a directory of files?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Iterating awk over a directory of files?
# 1  
Old 10-20-2014
Iterating awk over a directory of files?
I need to run the same
Code:
awk

function over an entire directly of files.
This is the
Code:
awk

:
Code:
awk '{$(NF+1)=1}1'

Is there a way that I can run this command once over all of the files, along the lines of:

Code:
awk '{$(NF+1)=1}1' *

so that I do not have to run this several times?
My main concern is how to output the results with a unique outfile name?
# 2  
Old 10-20-2014
awk can write to whatever file you please.

Code:
awk 'L && (L != FILENAME) { close(L) }
{ L=FILENAME }
{$(NF+1)=1 ; print > L ".new" }' *

This will create "filename.new" for each "filename" fed into it. FILENAME is a special variable in awk which is set to what you'd expect.

If you want it to work recursively, you can use find.
Last edited by Corona688; 10-20-2014 at 02:03 PM.. Reason: fix
This User Gave Thanks to Corona688 For This Post:
owwow14
# 3  
Old 10-20-2014
worked great! thanks !
This User Gave Thanks to owwow14 For This Post:
Corona688
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk separate files to one directory

I am trying to output all files that are made by this awk to a specific directory. awk -F '' '{f = $3 ".txt"; print > f}' input.txt Since the actual data has several hundred files I redirect the output (well tried to) to a directory. awk -F '' '{f = $3 ".txt"; close($3 ".txt")} print >... (6 Replies)
Discussion started by: cmccabe
6 Replies

2. Shell Programming and Scripting

Splitting Files with awk into other directory

I am trying to split into different files using awk: cat files | gawk '$1 ~ /---/ || $1 ~ /^deleting$/ || $1 ~ /^sorting$/ || $1 ~ /==/ {print}'| gawk '$1 ~ /---/ || $1 ~ /^deleting$/ || $1 ~ /^sorting$/ || $1 ~ /==/ {print}' |gawk '//{x="F"++i;}{print > x;}' What I am trying to do is make F*... (3 Replies)
Discussion started by: newbie2010
3 Replies

3. Shell Programming and Scripting

Using awk on multiple files in a directory

so i have a file system: /data/projects in this file system, there's about 300 files. on all files in this directory, i'm running: egrep -r 'Customer.*Processed' /data/projects/* is there an efficient (fast) awk way of searching through each file in the directory and providing an... (9 Replies)
Discussion started by: SkySmart
9 Replies

4. Shell Programming and Scripting

Iterating over files in a directory- windows machine

Dear all, I'm very new in scripting and using shell and prompt command. I have a program (called fasta) and I need to iterate it over all files in a folder, but I'm not able to do that. Pratically the fasta program wants a sintax like this: fasta.exe query_file.txt library_file.txt >... (2 Replies)
Discussion started by: gabrysfe
2 Replies

5. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

6. Shell Programming and Scripting

Iterating over a list using awk, when variable

Hi, I've recently started using Awk commands as i need it to pull information out of a file and give me the mean value of a series of numbers. Here is the code i run on my Infile and it works just fine. awk '{if ($1 == "Mam189") print $0}' Infile | awk '{if ($1 != $2) print $0}' | awk... (5 Replies)
Discussion started by: cavanac2
5 Replies

7. Shell Programming and Scripting

Iterating over subdirectories and dealing with files within them

Hello, I am working on a coding project for a class and to test the program I have created, I have come up with 100 different test cases. The program takes four text files as input, so each of the test cases is contained in a folder with four files. I have a folder called 'tests', within which... (1 Reply)
Discussion started by: dpryor
1 Replies

8. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

9. UNIX for Dummies Questions & Answers

Using Awk within awk to read all files in directory

I am wondering if anyone has any idea how to use an awk within awk to read files and find a match which adds to count. Say I am searching how many times the word crap appears in each files within a directory. How would i do that from the command prompt ... thanks (6 Replies)
Discussion started by: flevongo
6 Replies

10. Shell Programming and Scripting

bash Script: Issue with iterating Directory and store into array

Hi all, I am working on a backup based script, in which it enters to a directory and check the sub-directories and copy the names into an array. cd $CPFs k=0 for i in * do if then ARRs="$i" k=$(($k+1)) #echo "$i" ... (19 Replies)
Discussion started by: canishk
19 Replies
Login or Register to Ask a Question