How to use a loop for multiple files in a folder to run awk command?

Login or Register to Ask a Question and Join Our Community

How to use a loop for multiple files in a folder to run awk command?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Top Forums Shell Programming and Scripting How to use a loop for multiple files in a folder to run awk command?

04-09-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

How to use a loop for multiple files in a folder to run awk command?

Dear folks

I have two data set which there names are "final.map" and "1.geno" and look like this structures:
final.map:

Code:

gi|358485511|ref|NC_006088.3| 2044
 
 gi|358485511|ref|NC_006088.3| 2048
 gi|358485511|ref|NC_006088.3| 2187
 gi|358485511|ref|NC_006088.3| 17654
 
 gi|358485511|ref|NC_006088.3| 17666

1.geno:

Code:

gi|358485511|ref|NC_006088.3| 2048   G C 0 1 1
 gi|358485511|ref|NC_006088.3| 17654 A G 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 A G 0 1 1
 
 gi|358485511|ref|NC_006088.3| 17785 G A 0 1 1
 gi|358485511|ref|NC_006088.3| 30347 G C 1 1 2

Now, I am trying to run this command below:

Code:

awk -f example.awk 1.geno final.map > 1.dat

In this command "example.awk" contains the below command:
NR==FNR{a[$1,$2]=$5" "$6" "$7;next}{print $1,$2,a[$1,$2]?a[$1,$2]:"0 0 0"}

the output of the awk command give us "1.dat" which is

Code:

gi|358485511|ref|NC_006088.3| 2044   0 0 0
 
gi|358485511|ref|NC_006088.3| 2048   0 1 1
 
 gi|358485511|ref|NC_006088.3| 2187   0 0 0
 
 gi|358485511|ref|NC_006088.3| 17654 1 1 2
 
 gi|358485511|ref|NC_006088.3| 17666 0 1 1

My problem is I have around 300 *.geno files which I want to get *.dat out of the awk command. I am knowing the loop in unix could be helpful but I think I am using the loop wrong in some way.

Could any one give me an idea how to avoid to run awk each time separately and do it at one time by looping?

Last edited by vgersh99; 04-09-2015 at 02:07 PM.. Reason: code tags, please!

sajmar

View Public Profile for sajmar

Find all posts by sajmar

04-09-2015

Registered User

15,129, 5,008

Join Date: Jul 2012

Last Activity: 4 May 2020, 4:31 PM EDT

Location: Aachen, Germany

Posts: 15,129

Thanks Given: 735

Thanked 5,008 Times in 4,483 Posts

You forgot to mention which OS and shell you are using. With bourne type shells, this might work:

Code:

ls *.geno | while read FN; do awk -f example.awk $FN final.map > ${FN/geno/dat}; done

These 3 Users Gave Thanks to RudiC For This Post:

RudiC

View Public Profile for RudiC

Find all posts by RudiC

04-09-2015

Registered User

44, 0

Join Date: Sep 2013

Last Activity: 16 November 2016, 12:10 PM EST

Posts: 44

Thanks Given: 12

Thanked 0 Times in 0 Posts

Thank you so much Rudic. this command works well.

sajmar

View Public Profile for sajmar

Find all posts by sajmar

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Programming

Run sed and awk in multiple files in adirectory

Dear linux users I was running around of 200 djob for a Blastp search in a cluster. All my input files were protein fasta file (prot.fna.1, prot.fna.2 ...prot.fna.200). The output of each individual slurm job is located in a corresponding file ending with *test (prot.fna.1.test,...

2. UNIX for Dummies Questions & Answers

Loop awk command on files in a folder

Hi, I'd like to loop an action over all files with given extension within a folder. The "main" action is: awk -F "\t" 'BEGIN{OFS="\t"}{if ($10=="S") print$0; }' input.txt > output.txt The input.txt should be every file in the folder with *.subVCF extension; and the output should be a file...

3. Shell Programming and Scripting

FOR loop with multiple files as input and awk

Hi all , i want to pass multiple files as input to a for loop for i in file1 file2 file3 do some awk action < $i >> $i.out done but im getting error in that for loop is the way i use to pass files to awk using for correct and 2.we can directly pass multiple files to awk as...

4. Shell Programming and Scripting

Script for telnet and run one command kill it and run another command using while loop

( sleep 3 echo ${LOGIN} sleep 2 echo ${PSWD} sleep 2 while read line do echo "$line" PID=$? sleep 2 kill -9 $PID done < temp sleep 5 echo "exit" ) | telnet ${HOST} while is executing only command and exits.

5. Shell Programming and Scripting

Loop for row-wise averaging of multiple files using awk

Hello all, I need to compute a row-wise average of files with a single column based on the pattern of the filenames. I really appreciate any help on this. it would just be very difficult to do them manually as the rows are mounting to 100,000 lines. the filenames are as below with convention as...

6. Shell Programming and Scripting

Receiving 'ambiguous redirect' when trying to run command against multiple files

I came across the command string on https://www.unix.com/shell-programming-scripting/141885-awk-removing-data-before-after-pattern.html which was what I was looking for to be able to remove data before a certain pattern. However, outputting the result to a file seems to work on an individual basis...

7. Shell Programming and Scripting

using mv command for moving multiple files in a folder

Hi, I have a requirement where I need to move Bunch of folders containing multiple files to another archive location. i want to use mv command .I am thinking when we use mv command to move directory does it create directory 1st and then move all the files ? e.g source...

8. Shell Programming and Scripting

How to run multiple awk files

I'm trying some thing like this. But not working It worked for bash files Now I want some thing like that along with multiple input files by redirecting their outputs as inputs of next command like below Could you guyz p0lz help me on this #!/usr/bin/awk -f BEGIN { } script1a.awk...

9. UNIX for Dummies Questions & Answers

Foreach loop to run a perl script on multiple files

Hi, I have thousands of files in a directory that have the following 2 formats: 289620178.aln 289620179.aln 289620180.aln 289620183.aln 289620184.aln 289620185.aln 289620186.aln 289620187.aln 289620188.aln 289620189.aln 289620190.aln 289620192.aln.... and: alnCDS_1.fasta

10. Shell Programming and Scripting

How to run awk command having multiple lines

Hi, Can u see the code below. set xyz = `cat testt1.txt | awk '/-----/{\ print $1 }\ ' | tail -1` I need to execute it in c shell . What is wrong with the above command. When i write everything on a single line then it is working. Can anybody help me .

Login or Register to Ask a Question