Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

awk 2 files

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk 2 files
# 1  
Old 07-02-2014
awk 2 files
Hello All

file1
Code:
XXX 801;ZZZZZZZZZZ;XXX 5830;WWWWW11; 
XXX 801;ZZZZZZZZZZ;XXX 5829;FFFF1111; 
XXX 212;DDDDDDDDD;XXX 5855;GGGG1111; 
XXX 212;DDDDDDDDD;XXX 5854;HHHH222; 
XXX 212;DDDDDDDDD;XXX 212;YYYY1111; 
XXX 212;DDDDDDDDD;XXX 5853;UUUU3333; 
XXX 2710;FFFFFFFFF;XXX 20010;JJJJJ33333; 
XXX 2710;FFFFFFFFF;XXX 2710;LLLL2222;

file2
Code:
XXX 5853;1;QQQ5;UUUU3333;10752;NO 
XXX 2710;27;QQQ5;LLLL2222;10922;NO 
XXX 20010;28;QQQ5;JJJJJ33333;10922;NO 
XXX 212;2;QQQ5;YYYY1111;10752;NO 
XXX 5855;3;QQQ5;GGGG1111;10752;NO 
XXX 5854;4;QQQ5;HHHH222;10752;NO 
XXX 5830;6;QQQ5;WWWWW11;25600;NO 
XXX 5829;7;QQQ5;FFFF1111;25600;NO

output
Code:
XXX 801;ZZZZZZZZZZ;XXX 5830;WWWWW11;6;QQQ5;25600; 
XXX 801;ZZZZZZZZZZ;XXX 5829;FFFF1111;7;QQQ5;25600; 
XXX 212;DDDDDDDDD;XXX 5855;GGGG1111;3;QQQ5;10752; 
XXX 212;DDDDDDDDD;XXX 5854;HHHH222;4;QQQ5;10752; 
XXX 212;DDDDDDDDD;XXX 212;YYYY1111;2;QQQ5;10752; 
XXX 212;DDDDDDDDD;XXX 5853;UUUU3333;1;QQQ5;10752; 
XXX 2710;FFFFFFFFF;XXX 20010;JJJJJ33333;28;QQQ5;10922; 
XXX 2710;FFFFFFFFF;XXX 2710;LLLL2222;27;QQQ5;10922;

We have 2 files. Take column 4 (the field separator is ;) from file1 search in file2 column 4 (the field separator is ;) compare
if values are identical cut values from column 2, 3 and 5 (file2) and paste on the end of line in file1.

Can someone help with solution in awk?
# 2  
Old 07-02-2014
Code:
$ awk 'NR==FNR {f2[$4]=$2 OFS $3 OFS $5} NR!=FNR && f2[$4] {print $0 f2[$4]}' FS=';' OFS=';' file2 file1

# 3  
Old 07-02-2014
great working as should be, Thank you. could you please explain how exactly this is working?
# 4  
Old 07-02-2014
Code:
'NR==FNR {
   f2[$4]=$2 OFS $3 OFS $5
}

For lines in the first file specified on the command line (total record number equals file record number): Set an array ('f2') element indexed by the value of field 4 to the concatenated value of fields 2, 3 and 5 (separated by the output field separator OFS).
Code:
NR!=FNR && f2[$4] {
   print $0 f2[$4]
}'

For lines not in the first file and where there exists an element in array f2 for index of field 4 (i.e. this matches a line from the first file): Print the current line followed by the value of the array element (which we set earlier to fields 2/3/5 from the other file).

Code:
 FS=';' OFS=';' file2 file1

Set the field separator and output field separator to semi-colon, pass in the files (reversing the files is so we drive the actual output from file1, which preserves the ordering and saves messing with extra arrays)
This User Gave Thanks to CarloM For This Post:
vikus
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

2. Shell Programming and Scripting

awk, multiple files input and multiple files output

Hi! I'm new in awk and I need some help. I have a folder with a lot of files and I need that awk do something in each file and print a new file with the output. The input file name should be modified when I print the outpu files. Thanks in advance for help! :-) ciao (5 Replies)
Discussion started by: gabrysfe
5 Replies

3. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

4. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

5. UNIX for Dummies Questions & Answers

Using AWK: Extract data from multiple files and output to multiple new files

Hi, I'd like to process multiple files. For example: file1.txt file2.txt file3.txt Each file contains several lines of data. I want to extract a piece of data and output it to a new file. file1.txt ----> newfile1.txt file2.txt ----> newfile2.txt file3.txt ----> newfile3.txt Here is... (3 Replies)
Discussion started by: Liverpaul09
3 Replies

6. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

7. Shell Programming and Scripting

Awk : too many files 10

Hi, I m trying to split the one file into small segments based on userid field.I m getting the error "AWK : too many output files 10 record number 71". Can any one help me to resolve this issue or provide me someother code? Please note that, I don't have root admin previlege. I m login... (12 Replies)
Discussion started by: prabuk1
12 Replies

8. UNIX for Dummies Questions & Answers

Using Awk within awk to read all files in directory

I am wondering if anyone has any idea how to use an awk within awk to read files and find a match which adds to count. Say I am searching how many times the word crap appears in each files within a directory. How would i do that from the command prompt ... thanks (6 Replies)
Discussion started by: flevongo
6 Replies

9. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

10. Shell Programming and Scripting

Splitting input files into multiple files through AWK command

Hi, I needs to split *.txt files from single directory depends on the some mutltiple input values. i have wrote the code like below for file in *.txt do grep -i -h "value1|value2" $file > $file; done. My requirment is more input values needs to be given in grep; let us say 50... (3 Replies)
Discussion started by: arund_01
3 Replies
Login or Register to Ask a Question