Match the columns between 2 files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Match the columns between 2 files
# 1  
Old 08-13-2013
Match the columns between 2 files

I have two files I want to match ids in the 5th column of the file 1 with the first column of the file 2 and get the description for the matched ids as shown in the output

Code:
sno    nm    no    nm2    ID
1    cc    574372    yyyi    |6810|51234|
2    bb    119721    nmjk    |6810|51234|51179|    
3    ab    54161    mkkjn    |48193|46907|

Code:
ID        Description
6810    verygood_output
51234    Awesome work
46907    notbad
51179    excellent effort
48193    can do better

output format
Code:
sno    nm    no    nm2    Description    
1    cc    574372    yyyi    verygood_output;Awesome work
2    bb    119721    nmjk    verygood_output;Awesome work;excellent effort
3    ab    54161    mkkjn    notbad;can do better

# 2  
Old 08-13-2013
Try:
Code:
awk 'NR==FNR{for (i=2;i<=NF;i++) a[$1]=a[$1](i==2?"":" ")$i}NR!=FNR{for (i in a) gsub(i,a[i],$5);gsub("\\|",";",$5);gsub("^;|;$","",$5);print}' file2 file1


Last edited by bartus11; 08-13-2013 at 04:47 PM.. Reason: fixed version
# 3  
Old 08-13-2013
Code:
awk 'NR==FNR{match($0,"([^ ]+) +(.+)",x);_[x[1]]=x[2];next}FNR==1{$NF=_[$NF]}
FNR>1{n=split($NF,x,"|");$NF="";for(i=1;++i<n;){z=_[x[i]];$NF=$NF?$NF";"z:z}}1' file2 file1

--ahamed
# 4  
Old 08-14-2013
This one makes an effort to keep the spacing
Code:
awk 'NR==FNR {
  id=$1
  sub("^[[:alnum:]]+[[:space:]]+","")
  a[id]=$0
  next
}
{
  n=split($NF,b,"|")
  sub("[[:alnum:]|]+ *$","")
  printf "%s",$0
  sep=""
  for (i=1; i<=n; ++i) if (b[i] in a) {printf "%s",sep a[b[i]]; sep=";"}
  print ""
}' file2 file1

# 5  
Old 08-15-2013
Another option:
Code:
awk '
  NR==FNR{
    A[$1]=$2
    next
  } 
  FNR==1{
    sub("ID",A["ID"])
  }
  {
    for(i=2; i<NF; i++) $i=A[$i]
    sub(/;/,x)
    sub(/; *$/,x)
  }
  1
' FS='   +' file2 FS=\| OFS=\; file1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Data match 2 files based on first 2 columns matching only and join if match

Hi, i have 2 files , the data i need to match is in masterfile and i need to pull out column 3 from master if column 1 and 2 match and output entire row to new file I have tried with join and awk and i keep getting blank outputs or same file is there an easier way than what i am... (4 Replies)
Discussion started by: axis88
4 Replies

2. Shell Programming and Scripting

Comparing two columns in two files and printing a third based on a match

Hello all, First post here. I did not notice a previous post to help me down the right path. I am looking to compare a column in a CSV file against another file (which is not a column match one for one) but more or less when a match is made, I would like to append a third column that contains a... (17 Replies)
Discussion started by: dis0wned
17 Replies

3. UNIX for Dummies Questions & Answers

Match the columns between two files and output

Hi Help, I have two files namely a.txt and b.txt a.txt looks like a.txt 1 2 2 1 3 3 2 4 4 4 5 6 6 7 7 b.txt looks like, b.txt 1 2 1 1 3 2 2 4 3 3 4 4 4 5 5 (2 Replies)
Discussion started by: Indra2011
2 Replies

4. Shell Programming and Scripting

Match first two columns and average third from multiple files

I have the following format of input from multiple files File 1 24.01 -81.01 1.0 24.02 -81.02 5.0 24.03 -81.03 0.0 File 2 24.01 -81.01 2.0 24.02 -81.02 -5.0 24.03 -81.03 10.0 I need to scan through the files and when the first 2 columns match I... (18 Replies)
Discussion started by: ncwxpanther
18 Replies

5. Shell Programming and Scripting

Return first two columns if match found among two files

Hi, I have FileA with one column. File B with 15 columns separated by comma delimiter. I need to compare the FILEA value with all 15 columns of FILEB... if matches, need to return the 1st, 2nd column values of FILEB. How to achieve this through shell script? Thanks in advance. (5 Replies)
Discussion started by: vamsikrishna928
5 Replies

6. Shell Programming and Scripting

Match two columns from two files and print output

Hello, I have two files which are of the following format File 1 which has two columns Protein_ID Substitution NP_997239 T53R NP_060668 V267M NP_058515 P856A NP_001206 T55M NP_006601 D371Y ... (2 Replies)
Discussion started by: nans
2 Replies

7. Shell Programming and Scripting

Match files based on either of the two columns awk

Dear Shell experts, I have 2 files with structure: File 1: ID and count head test_GI_count1.txt 1000094 2 10039307 1 10039641 1 10047177 11 10047359 1 1008555 2 10120302 1 10120672 13 10121776 1 10121865 32 And 2nd file: head Protein_gi_GeneID_symbol.txt protein_gi GeneID... (11 Replies)
Discussion started by: smitra
11 Replies

8. Shell Programming and Scripting

Match columns several files

Hey fellas! Here come my problem. I appreciate if you have a look at it. I have several files with following structure: file_1:1 21 4 45 file_2:2 31 4 153 6 341 and so on... and I have a 'reference' file look like this: File_ref:A 1 B 2 C 3 (5 Replies)
Discussion started by: @man
5 Replies

9. UNIX for Dummies Questions & Answers

Two files; if cells match then copy over other columns

My current issue is dealing with two space delimited files. The first file has column 1 as the sample ID's, then columns 2 - n as the observations. The second file has column 1 as the sample ID's, column 2 as the mother ID's, column 3 as the father ID's, column 4 as the gender, and column 5... (3 Replies)
Discussion started by: Renyulb28
3 Replies

10. Shell Programming and Scripting

Match strings in two files and compare columns of both

Good Morning, I was wondering if anybody could tell me how to achieve the following, preferably with a little commenting for understanding. I have 2 files, each with multiple rows with multiple columns. I need to find each row where the value in column 1 of file 1 matches column 1... (10 Replies)
Discussion started by: GarciasMuffin
10 Replies
Login or Register to Ask a Question

Featured Tech Videos