Looping through pairs of files with awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Looping through pairs of files with awk
# 1  
Old 07-02-2013
Looping through pairs of files with awk

Hi all,

please help me construct the command. i want to loop through all files named bam* and bed*. My awk works for a particular pair but there are too many pairs to do manually.

I have generated multiple files in a folder in a given pattern. The files are named like

Code:
bam_fixed1.bam
bed_fixed1.bed
bam_fixed2.bam
bed_fixed2.bed
....
.....

so that every bam file is paired with a bed file. The fixed portion of the filename is the same for a particular pair of bam and bed.

I want to construct an awk command which acts on a bam and its particular bed pair..

what I want to do is for all pairs

Code:
awk 'NR == FNR { commands }'  bam_fixed1.bam bed_fixed1.bed > out_fixed1.txt

awk 'NR == FNR { commands }' bam_fixed2.bam bed_fixed2.bed > out_fixed2.txt

...

If there was only 1 file I can use a for loop like

Code:
for f in bam_*.bam
do
  commands with $f
done

How do I do this for multiple files and corresponding pairs?
# 2  
Old 07-02-2013
One way to do this:

Code:
for f1 in bam_*.bam
do
    mid=${f1#*_}
    mid=${mid%.*}
    f2=bed_${mid}.bed
    awk 'NR==FNR { commands }' $f1 $f2 > out_${mid}.txt
done

This User Gave Thanks to balajesuri For This Post:
# 3  
Old 07-02-2013
thanks , this is great !!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Concatenate files by pairs

Hi, I would like to batch concatenate files by pairs. I have quite a few of them so I would not like to do that pair by pair separately. the names of the file is of the type: file1.fastq newfile1_new.fastq file2.fastq newfile2_new.fastq and so on... I would like to concatenate file1... (2 Replies)
Discussion started by: jawad
2 Replies

2. Shell Programming and Scripting

Help on looping using awk

I have the data like this: PONUMBER,SUPPLIER,LINEITEM,SPLITLINE,LINEAMOUNT,CURRENCY IR5555,Supplier1,1,1,83.1,USD IR5555,Supplier1,1,3,40.4,USD IR5555,Supplier1,1,6,54.1,USD IR5555,Supplier1,1,8,75.1,USD IR5556,Supplier2,1,1,41.1,USD IR5556,Supplier2,1,3,43.1,USD ... (3 Replies)
Discussion started by: jeffreybsu
3 Replies

3. Shell Programming and Scripting

Looping through files in pairs

Hi all, Please guide. It has to do with parsing the input file names. I have a fairly large number of files, I want to do some operations on them in a pairwise fashion (every file has a pair). The names are in the following pattern, with the pairs of files named with _1 and _2 , the... (4 Replies)
Discussion started by: newbie83
4 Replies

4. Shell Programming and Scripting

Extracting key/value pairs in awk

I am extracting a number of key/value pairs in awk using following: awk ' /xyz_session_id/ { n=index($0,"xyz_session_id"); id=substr($0,n+15,25); a=$4; } END{ for (ix in a) { print a } }' I don't like this Index + substr with manually calculated... (5 Replies)
Discussion started by: migurus
5 Replies

5. UNIX Desktop Questions & Answers

trying to cat multiple pairs of files

I have a number of files in a directory named like this: fooP1, fooN1, fooP2, fooN2 ... fooP(i), fooN(i). I'd like to know how to combine each P and N pair into a single file, foo(i) TIA John Balwit (1 Reply)
Discussion started by: balwit
1 Replies

6. UNIX for Dummies Questions & Answers

awk code to process column pairs and find those with more than 1 set of possible values

Hi, I have a very wide dataset with pairs of columns starting from column 7 onwards (see example below). 0 123456 -1 0 0 -9 0 0 1 2 2 2 1 1 1 1 2 2... 0 123457 -1 0 0 -9 1 2 1 1 2 2 0 0 0 0 2 2... 0 123458 -1 0 0 -9 0 0 1 2 2 2 1 1 2 2 1 2... 0 123459 -1 0 0 -9 1 2 0 0 2 2 1 1 1 2 1 1...... (2 Replies)
Discussion started by: kasan0
2 Replies

7. Shell Programming and Scripting

looping in awk

How do I remove last comma? echo "xx yy zz" | awk 'BEGIN{FS=" "}{for (i=1; i<=NF; i++) printf "%s,", $i}'output: xx,yy,zz, required output: xx,yy,zz or (ideally!): xx, yy & zz many thanks in advance! (4 Replies)
Discussion started by: euval
4 Replies

8. Shell Programming and Scripting

Matching Pairs with AWK or Shell

Hello, I have a file Pairs.txt composed of 3 columns. For simplicity, this is the file: A B C B D W X Y Z Z W K B A C Y J Q F M P Y X Z I am trying to reduce the file such that I only keep the first occurrence of any "pair". In the example above, I would only want to keep one... (6 Replies)
Discussion started by: InfoSeeker
6 Replies

9. UNIX for Dummies Questions & Answers

Help with AWK looping

I'm trying to parse a configuration text file using awk. The following is a sample from the file I'm searching. I can retrieve the formula and recipe names easily but now I want to take it one step farther. In addition to the formula name, I would like to also get the value of the attribute... (6 Replies)
Discussion started by: new2awk
6 Replies

10. UNIX for Advanced & Expert Users

Looping in awk

Can somebody give me a cleaner way of writing the following script. I was thinking that I could use a loop in the awk statement. It works fine the way it is but I just want the script to be cleaner. #!/usr/bin/sh for r in 0 1 2 3 4 5 6 do DAY=`gdate --date="${r} days ago" +%m\/%d\/%y`... (3 Replies)
Discussion started by: keelba
3 Replies
Login or Register to Ask a Question