UNIX for Dummies Questions & Answers

Well, pretty obvious that you are going to need to read through each file at least once; but there is the issue - to only have to read through each file once.

My thought would be to do something like the following (in loose coding)
Notes:
In the first pass, since you said each range was 36, I just wrote to do all thirty-six manually. I suppose you could augment and have it loop through from the first parameter $1 to the second parameter $2
In the second pass, I check to see if the second parameter has its array value been already set. If not, then I print the two parameters of the second file.

Code:

awk {
  FILENAME=file1
  valid[$1]=1
  valid[$1+1]=1
  ...
  valid[$1+35]=1

  FILENAME=file2
  if valid[$2]!=1
    then
       print $1 $2
  fi
} file1 file2

Hi,

Thanks for the reply but I can´t seem to get it to work. Here is what I did and the error message that it posted:

[jofa@em64node02 ~]$ awk {
FILENAME=/mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_perfectReads_trimmed_start_end.txt
valid[$1]=1
awk: cmd. line:1: valid[$1+1]=1
{
awk: cmd. line:1: ^ unexpected newline or end of string
valid[$1+3]=1
[jofa@em64node02 ~]$ FILENAME=/mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_perfectReads_trimmed_start_end.txt
[jofa@em64node02 ~]$ valid[$1]=1
[jofa@em64node02 ~]$ valid[$1+1]=1
[jofa@em64node02 ~]$ valid[$1+2]=1
[jofa@em64node02 ~]$ valid[$1+3]=1
[jofa@em64node02 ~]$ valid[$1+4]=1
[jofa@em64node02 ~]$ valid[$1+5]=1
[jofa@em64node02 ~]$ valid[$1+6]=1
[jofa@em64node02 ~]$ valid[$1+7]=1
[jofa@em64node02 ~]$ valid[$1+8]=1
[jofa@em64node02 ~]$ valid[$1+9]=1
[jofa@em64node02 ~]$ valid[$1+10]=1
[jofa@em64node02 ~]$ valid[$1+11]=1
[jofa@em64node02 ~]$ valid[$1+12]=1
[jofa@em64node02 ~]$ valid[$1+13]=1
[jofa@em64node02 ~]$ valid[$1+14]=1
[jofa@em64node02 ~]$ valid[$1+15]=1
[jofa@em64node02 ~]$ valid[$1+16]=1
[jofa@em64node02 ~]$ valid[$1+17]=1
[jofa@em64node02 ~]$ valid[$1+18]=1
[jofa@em64node02 ~]$ valid[$1+19]=1
[jofa@em64node02 ~]$ valid[$1+20]=1
[jofa@em64node02 ~]$ valid[$1+21]=1
[jofa@em64node02 ~]$ valid[$1+22]=1
[jofa@em64node02 ~]$ valid[$1+23]=1
[jofa@em64node02 ~]$ valid[$1+24]=1
[jofa@em64node02 ~]$ valid[$1+25]=1
[jofa@em64node02 ~]$ valid[$1+26]=1
[jofa@em64node02 ~]$ valid[$1+27]=1
[jofa@em64node02 ~]$ valid[$1+28]=1
[jofa@em64node02 ~]$ valid[$1+29]=1
[jofa@em64node02 ~]$ valid[$1+30]=1
[jofa@em64node02 ~]$ valid[$1+31]=1
[jofa@em64node02 ~]$ valid[$1+32]=1
[jofa@em64node02 ~]$ valid[$1+33]=1
[jofa@em64node02 ~]$ valid[$1+34]=1
[jofa@em64node02 ~]$ valid[$1+35]=1
[jofa@em64node02 ~]$ FILENAME=/mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_var_Readpos.txt
[jofa@em64node02 ~]$ if valid[$2]!=1
> then
> print $1 $2
> fi
-bash: valid[]!=1: command not found
[jofa@em64node02 ~]$ } /mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_perfectReads_trimmed_start_end.txt /mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_var_Readpos.txt > /mgs/projects/sequencing/joao/cow_Solexa_16.31SDM/Solexa_data/NovoCraft/cow31_realVar.txt
-bash: syntax error near unexpected token `}'
[jofa@em64node02 ~]$

I was just proposing a thought process to solve the problem using awk and an array within awk. I was not trying to deal with all of the syntax of awk; hence my "in loose coding".

If this apprach seems reasonable, then take a look at the syntax of awk (or maybe someone else might propose the correct syntax) and see what you come up with. I would suggest you start with a couple of small test files to verify proper programming, and NOT to do a first test against those two very large files.