Login or Register to Ask a Question and Join Our Community


UNIX for Dummies Questions & Answers

Loop awk command on files in a folder

 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Loop awk command on files in a folder
# 1  
Old 11-06-2015
Loop awk command on files in a folder
Hi, I'd like to loop an action over all files with given extension within a folder.

The "main" action is:
Code:
 awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt

The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file named in the same way than the input file, but with addition of the string 'NEW'

I am currently trying:

Code:
for i in $(ls *.subVCF); do echo $i; a=$(echo $i | awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }'); $i > 'NEW'$a; done

But it isn't working... what I am doing wrong?

Thanks in advance
# 2  
Old 11-06-2015
WHAT is not working - post the error msgs!

---------- Post updated at 14:41 ---------- Previous update was at 14:37 ----------

And, you don't need the command substitution in the for loop. And, you seem to terminate the second command substitution too early - the awk doesn't have input nor output files (actually, it works on stdin which is the pipe from echo $i).

awk doesn't need the for loop - it can work on a series of input files by itself.
# 3  
Old 11-06-2015
Sorry, I should have mentioned, "doesn't work" meant that I got empty files :/

Also, I got it working, thanks to one of your previous replies to another post (How to use a loop for multiple files in a folder to run awk command?)

Code:
ls *.subVCF | while read FN; do awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }'  $FN > ${FN/subVCF/txt}; done

# 4  
Old 11-06-2015
Try:-
Code:
awk -F'\t' 'FNR==1{ofile="NEW" FILENAME}$10=="S"{print > ofile}' *.subVCF

This User Gave Thanks to Yoda For This Post:
dovah
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

For loop to accept params and delete folder/files

Hi Folks - I'm trying to build a simple for loop to accept params and then delete the folder & files on the path older than 6 days. Here is what I have: Purge () { for _DIR in "$1" do find "${_DIR}"/* -mtime +0 -exec rm {} \; done } I would be passing... (4 Replies)
Discussion started by: SIMMS7400
4 Replies

2. Shell Programming and Scripting

Apply command to all files in folder

Hi all! I have this command grep -E '^\To: |^\Date: |^\Subject: ' fileA.txt > fileA_1.txt && grep -v '^\To: |^\Date: |^\Subject: ' fileA.txt >> fileA_1.txt && rm fileA.txt && sed -i -e 's/\(Date: \|Subject: \|To: \)//g' fileA_1.txtHow do I apply it to all the files in the folder (each file has a... (7 Replies)
Discussion started by: guilliber
7 Replies

3. Shell Programming and Scripting

awk error when increasing number of files in folder

I have a folder with several files of which I want to eliminate all of the terms that they have in common using `awk`. Here is the script that I have been using: awk ' FNR==1 { if (seen++) { firstPass = 0 outfile = FILENAME "_new" ... (4 Replies)
Discussion started by: owwow14
4 Replies

4. Shell Programming and Scripting

How to use a loop for multiple files in a folder to run awk command?

Dear folks I have two data set which there names are "final.map" and "1.geno" and look like this structures: final.map: gi|358485511|ref|NC_006088.3| 2044 gi|358485511|ref|NC_006088.3| 2048 gi|358485511|ref|NC_006088.3| 2187 gi|358485511|ref|NC_006088.3| 17654 ... (2 Replies)
Discussion started by: sajmar
2 Replies

5. Shell Programming and Scripting

For loop for number of files in a folder

Hi All, Need a for loop which should run for number of files in a folder and should pass the file name as parameter to another shell script for each loop. Please help me. Thanks. (2 Replies)
Discussion started by: chillblue
2 Replies

6. UNIX for Dummies Questions & Answers

how to create a command that would print files only from 1 folder

Hi there how to create a command in csh that would print files only from 1 folder but as an argument takes home directory for e.g. in my home diecrtory I have 3 folders unix, windows,mac and alot of files. and I running the program as ./op ~ this should return me files from unix folder without... (1 Reply)
Discussion started by: FUTURE_EINSTEIN
1 Replies

7. Shell Programming and Scripting

loop through files in each folder and perform sed

Dear all, I have few log folders in directory (FILE) and i need to perform sed on each files inside each folders. Here is my script, and i wish to rename each file back to the same file name after modification. Can anyone guide me on my script below? eg: folder1 contain files... (2 Replies)
Discussion started by: ymeyaw
2 Replies

8. Shell Programming and Scripting

Loop through text file > Copy Folder > Edit XML files in bulk?

I have a text file which contains lines in this format - it contains 105 lines in total, but I'm just putting 4 here to keep it short: 58571,east_ppl_ppla_por 58788,east_pcy_hd_por 58704,east_pcy_ga_por 58697,east_pcy_pcybs_por It's called id_key.txt I have a sample folder called... (9 Replies)
Discussion started by: biscuitcreek
9 Replies

9. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

10. Shell Programming and Scripting

using mv command for moving multiple files in a folder

Hi, I have a requirement where I need to move Bunch of folders containing multiple files to another archive location. i want to use mv command .I am thinking when we use mv command to move directory does it create directory 1st and then move all the files ? e.g source... (4 Replies)
Discussion started by: rkmbcbs
4 Replies
Login or Register to Ask a Question