Visit The New, Modern Unix Linux Community


AWK Script For Merging Text Files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting AWK Script For Merging Text Files
# 1  
AWK Script For Merging Text Files

Hello,

I am trying to merge data from two text files. One file (File1) contains a listing of data which includes the trial number in Column 5, while the other text file (File2) contains what category the trial belongs to.

Here is a snippet of what File1 looks like.

Code:
1 Arrow_ST 9.738 0.905 1
1 Arrow_Gamble 10.703 0.188 1
1 Gamble_Loss 14.247 0.292 2
1 Gamble_Win 14.578 0.151 2

And here is a snippet of what File2 looks like:

Code:
1 Gamble
2 SureThing

In other words, for all of Trial 1 (i.e., the first two snippet lines of File1) belong to the category "Gamble", all of Trial 2 (i.e., the last two snippet lines of File2) belong to the category "SureThing", and so on.

What I want to do is merge the two, so that an additional column is created in File1 that lists the category membership for each trial. So, my output should look as follows:
Code:
1 Arrow_ST 9.738 0.905 1 Gamble
1 Arrow_Gamble 10.703 0.188 1 Gamble
1 Gamble_Loss 14.247 0.292 2 SureThing
1 Gamble_Win 14.578 0.151 2 SureThing

Any help or nudges in the right direction would be greatly appreciated. Thanks!

-Jahn

Last edited by radoulov; 04-24-2011 at 04:28 PM.. Reason: Code tags, please!
# 2  
Code:
awk 'NR == FNR { 
  cat[$1] = $2; next 
  }
{ 
  print $0, cat[$5]
  }' file2 file1

# 3  
Code:
#!/bin/bash


awk 'BEGIN{
while ( ( getline < "file2" ) > 0){
   array_test[$1]=$0
}
}


{
print $1 " " $2 " " $3 " " $4 " " array_test[$5]
}' file1


a simple use of associative arrays Smilie

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #138
Difficulty: Easy
The Unix philosophy is a computer operating system idea that means the operating system provides a set of basic tools that perform a limited, well-defined function, a unified filesystem as the main method of communication, and a shell scripting and command language to combine modular tools to perform more complex workflows.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Comparing and merging two text files

Hey everybody, I am new here and already a question to ask, I just recently started some bioinformatic work for my PhD so I am slowly learning Anyway, here is my problem, I have two text files, one contains the complete data file with 43000 genes and their read counts for all my samples... (1 Reply)
Discussion started by: ant55
1 Replies

2. UNIX for Dummies Questions & Answers

Merging two text files by two columns

Hi, I have two text files that I would like to merge/join. I would like to join them if the first columns of both text files match and the second column of the first text file matches the third column of the second text file. Example input: First file: 1334 10 0 0 1 5.2 1334 12 0 0 1 4.5... (4 Replies)
Discussion started by: evelibertine
4 Replies

3. UNIX for Dummies Questions & Answers

Merging two text files by a column

I have two text files. One has two columns and looks like below: rs# otherallele_freq rs10399749 0 rs4030303 0 rs4030300 0 rs940550 1.000 rs13328714 0 rs11490937 0 rs6683466 0 rs12025928 1.000 rs6650104 0 rs11240781 0... (5 Replies)
Discussion started by: evelibertine
5 Replies

4. UNIX for Dummies Questions & Answers

Merging two text files by a column

So I have two text files. The first one looks like this: refsnp_id chr_name chrom_start 1 rs1000000 12 126890980 2 rs10000010 4 21618674 3 rs10000012 4 1357325 4 rs10000013 4 37225069 5 rs1000002 3 183635768 And the second one looks like this: AUC rs1000000 0.03 0.1240 AUC ... (4 Replies)
Discussion started by: evelibertine
4 Replies

5. UNIX for Dummies Questions & Answers

Merging two text files with variable coloumns

Hi All, I have two text files containing space delimited columns. The first file contains 9 columns and the second one contain 3 columns. I want to copy the 3 coloumns from the 2nd file and paste them in 1st file after 9 coloumns. Ex. File1.txt contains 9 coloumns C1 C2 C3 C4 C5 C6 C7 C8 C9... (6 Replies)
Discussion started by: Unilearn
6 Replies

6. Shell Programming and Scripting

merging text files into one

Hi good day Is there a shell script I can use to join multiple text files in a folder and put the name(excluding the extension) of the text file before it's contents and put EOF at the end of each portion. so for example, say I have file1.dat, file2.dat, file3.dat i'll get file1 {text... (7 Replies)
Discussion started by: johnstrong
7 Replies

7. UNIX for Dummies Questions & Answers

Merging all text files in the order which they recieved

Hi guru's :) I am new to UNIX. requirement is merging all text files from one folder in the order which they have received. EX: under /Test folder files received in certain intervals. abc.txt @ 1:00 AM 123.txt @ 2:00 AM xyz.txt @ 4:00 AM . . . 456.txt @ 10:00 PM ... (4 Replies)
Discussion started by: arund_01
4 Replies

8. Shell Programming and Scripting

Merging all text files in the order which they recieved

Hi There, I am new to UNIX. requirement is merging all text files from one folder in the order which they have received. EX: under /Test folder files received in certain intervals. abc.txt @ 1:00 AM 123.txt @ 2:00 AM xyz.txt @ 4:00 AM . . . 456.txt @ 10:00 PM so need to develop a... (3 Replies)
Discussion started by: arund_01
3 Replies

9. Shell Programming and Scripting

merging text files

hi, i need to merge 2 lakh text files ..... can somebody please help me with a script/program for it... (8 Replies)
Discussion started by: code19
8 Replies

10. Shell Programming and Scripting

merging few columns of two text files to a new file

hi i need to select a few columns of two txt files and write it to a new file. there is one common field for both of these files. plz help me in this thanks in advance (4 Replies)
Discussion started by: kolvi
4 Replies

Featured Tech Videos