File comparison and proccessing using awk


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting File comparison and proccessing using awk
# 1  
Old 05-13-2014
File comparison and proccessing using awk

Hi Guys,

I am having two requirement in one of my scripts. please help out to find a fast solution using AWK (since there is lot of data to be processed)

1) First snippet -

File1 has two columns and file2 has three columns

If any value of column 1 of file1 matches with column 1 of file2 then output file ie file3.csv should have corresponding value of Columns 1 & 2 of file1 with corresponding values of column 3 & 4 of file2.

For eg :

File1.txt

Code:
10000000    |25000000
20000000    |35000000

File2.txt

Code:
10000000    | 0000001.99|R
20000000    | 0000002.99|P

file3.csv

Code:
"10000000    ","25000000"," 0000001.99","R "
"20000000    ","35000000"," 0000002.99","P "

2) Second snippet
Second script should just convert input.txt to output.csv after doing little proccessing as shown below:

input.txt

Code:
10000000    | 0000001.99|R
20000000    | 0000002.99|P

output.csv

Code:
"10000000    "," "," 0000001.99","R "
"20000000    "," "," 0000002.59","P "

(The second column needs to be left blank as shown above ie between " ", so that second column would appear blank while opening in an excel Smilie )

Thanks in Advance !!!
Moderator's Comments:
Mod Comment Code tags for code, please.

Last edited by Corona688; 05-13-2014 at 02:54 PM..
# 2  
Old 05-13-2014
What have you tried?
# 3  
Old 05-14-2014
I tried using loops, but it is not fast. I am not familiar with AWK. If someone could tell me how to implement it using AWK it would be much helpful
# 4  
Old 05-14-2014
Try:
Code:
awk -F"|" 'NR==FNR{col1[$1]=$1;col2[$1]=$2} NR>FNR && $1 in col1{printf("\"%s\",\"%s\",\"%s\",\"%s\"\n",col1[$1],col2[$1],$1,$2)}' file1 file2

for the first part.

For info on awk go here.Awk - A Tutorial and Introduction - by Bruce Barnett
This User Gave Thanks to chacko193 For This Post:
# 5  
Old 05-19-2014
Thank you..

Quote:
Originally Posted by chacko193
Try:
Code:
awk -F"|" 'NR==FNR{col1[$1]=$1;col2[$1]=$2} NR>FNR && $1 in col1{printf("\"%s\",\"%s\",\"%s\",\"%s\"\n",col1[$1],col2[$1],$1,$2)}' file1 file2

for the first part.

For info on awk go here. Awk - A Tutorial and Introduction - by Bruce Barnett
Hi Chacko193,
The awk you have suggested is working as expected.

Thank you !! Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

File comparison using awk

Hi All, i have two files file1 ,file 2 file 1 col1|col2|col3|col4|col5|col6|col7|col8 11346925|0|2009-09-20|9999-12-31|100|0 11346925|0|2009-09-20|9999-12-31|120|0 12954311|0|2009-09-11|9999-12-31|100|0 12954311|0|2009-07-23|2999-12-31|120|0 12954312|0|2009-09-11|9999-12-31|100|0... (9 Replies)
Discussion started by: mohanalakshmi
9 Replies

2. Shell Programming and Scripting

File comparison using awk

my files are as follows fileA sepearated by tab /t 00 lieferungen 00 attractiop 01 done 02 forness 03 rasp 04 alwaysisng 04 funny 05 done1 fileB funnymou120112 funnymou234470 mou3raspnhdhv rddfgmoudone1438748 so all those record which are greater than 3 and which are not... (6 Replies)
Discussion started by: rajniman
6 Replies

3. Shell Programming and Scripting

awk file comparison, x lines after matching as output

Hello, I couldn't find anything on the Forum that would help me to solve this problem. Could any body help me process below data using awk? I have got two files: file1: Worker1: Thomas Position: Manager Department: Sales Salary: $5,000 Worker2: Jason Position: ... (5 Replies)
Discussion started by: killerbee
5 Replies

4. Shell Programming and Scripting

Urgent Help Required for File Comparison using Awk

Hello All, I am having a below requirement. File1 contains KEY|VIN|SEQUENCE|COST 101 | XXX111 | 1 | 234.22 234 | XXX111 | 2 | 134.32 444 | ABC234 | 1 | 100.22 555 | DFF611 | 1 | 734.82 FILE 2 Contains only VINs XXX111 DFF611 Now if the VIN from file 1 is present in... (8 Replies)
Discussion started by: dinesh1985
8 Replies

5. Shell Programming and Scripting

awk column comparison big file

Hi all, I would like to compare a column in one file to a column in another file and when there is a match it prints the first column and the corresponding second column. Example File1 ABA ABC ABE ABF File 2 ABA 123 ABB 124 ABD 125 ABC 126 So what I would like printed to a... (6 Replies)
Discussion started by: pcg
6 Replies

6. Shell Programming and Scripting

Text Proccessing with sort,uniq,awk

Hello, I have a log file with the following input: X , ID , Date, Time, Y 01,01368,2010-12-02,09:07:00,Pass 01,01368,2010-12-02,10:54:00,Pass 01,01368,2010-12-02,13:07:04,Pass 01,01368,2010-12-02,18:54:01,Pass 01,01368,2010-12-03,09:02:00,Pass 01,01368,2010-12-03,13:53:00,Pass... (12 Replies)
Discussion started by: rollyah
12 Replies

7. Shell Programming and Scripting

awk comparison

Hello all, Probably a very simple question, I am stuck with a small part of a code: I am trying to do a comparison to get the maximum value of column 6 if columns 1, 4 and 5 of two or more rows match. Here is what I am doing: awk -F'\t' '{if ($6 > a)a=$6}END{for (i in a) print i"\t"a}' ... (4 Replies)
Discussion started by: jaysean
4 Replies

8. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

9. Shell Programming and Scripting

Looking for AWK Solution for column comparison in a single file

- I am looking for different kind of awk solution which I don't think is mentioned before in these forums. Number of rows in the file are fixed Their are two columns in file1.txt 1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10 I am looking for 3... (1 Reply)
Discussion started by: softwarekids23
1 Replies

10. Shell Programming and Scripting

awk program for file comparison

Hello there, I'm trying to write an awk program in bash shell with the following three input files: File 1 1001 1 2 3 1002 4 5 6 1003 7 8 9 1004 10 11 12 File 2 1001 11 22 33 1002 44 55 66 1004 100 111 122 ... (4 Replies)
Discussion started by: kbirde
4 Replies
Login or Register to Ask a Question