Shell Programming and Scripting

Hi everyone,

I got a problem with merging files and hoped one of you would have an idea how to approach this issue. I tried it with awk, but didn't get far. This is what I have:

I got 40 files looking like the ones below. All have three columns but the number of rows differs (20000 to 50000).

eg. file1

....

eg. file2

....

I would now need to merge them according to their genome coordinates (ie 'chromosome' and 'position_on_chromosome' -both infos together give the coordinates). All coordinates (column 1 & 2) should be listed, if present in one file or in all files (=complete list). The third columns of the original files should be added after each other.


This is how it should look like:

.....


A bit complicated to explain, but I hope you got what I mean

Any help would be greatly appreciated!

Edit note: ...just saw now, that it doesn't leave the space in the output table for those 'x' which are empty. Replaced the space (empty cell in the table) with a 'e' for clarification.

Why in your output those lines are missing:

Code:

chr5 173197909 x
chr5 173418499 x
chr5 172800339 x
chr5 172800347 x
chr5 172805352 x
chr5 172805379 x

?

sorry, you're of course right. I just didn't post the complete list as it was just for illustration purpose. Corrected it now. Thanks.

One more thing... shouldn't there be line like this for entries that are identical in both files:

Code:

chr1 631246 x x

instead of

Code:

chr1 631246 x e
chr1 631246 e x

?

Of course, you're right. This is actually exactly the problem I have. To merge the files so that I only get one entry per position including all additional info.
Sorry, accidentally pasted the wrong output (where I exactly don't get that). Thanks for noticing! Corrected now.

Try this script:

Code:

#!/usr/bin/perl
for $f (@ARGV){
  open I, "$f";
  while (<I>){
    chomp;
    @F=split / /;
    $h="$F[0] $F[1]" if $n==0;
    $h.=" $F[2]" if $.==1;
    $n++ if $.==1;
    $a{"$F[0] $F[1]"}=$a{"$F[0] $F[1]"}?$a{"$F[0] $F[1]"} . " $F[2]":"e " x ($n-1) . "$F[2]" if $.>1;
    if (eof){
      $.=0;
      for $i (keys %a){
        $a{$i}="$a{$i} e" if length $a{$i}<($n*2-1);
      }
    }
  }
}
print "$h\n";
for $i (keys %a){
  print "$i $a{$i}\n";
}

Run it like this: ./script.pl file1 file2 file3 ...

...the script works absolutely fantastic!

many thanks for this!!