Compare fields in two files line by line


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare fields in two files line by line
# 1  
Old 09-22-2013
Compare fields in two files line by line

I am new to awk scripting.

I want to do a field by word (field) comparison of two files File1.txt and File2.txt.

The files contain a list of | (pipe) separated field.

**File 1:
-------------------
Code:
aaa|bbb|ccc|eee|fff
lll|mmm|nnn|ooo|ppp
rrr|sss|ttt|uuu|vvv**

File 2:
-------------------
Code:
aaa|bbb|ccc|eee|fff
rrr|sss|ttt|uuu|vvv
rrr|sss|ttt|uuu|uuu

We compare the same line no. in both the files.
Fields in Line 1 of both file match.

In Line 2 all the fields (lll, mmm, nnn, ooo, ppp) donot not match with all fields (rrr, sss, ttt, uuu, vvv) in line 2 of File 2. Similarly the 5th field of 3rd line in both the files donot match.

Hence Line no. 2 and Line no. 3 should get echoed by bash.

Both files will follow an order.

Last edited by Don Cragun; 09-22-2013 at 01:38 PM.. Reason: Add CODE tags
# 2  
Old 09-22-2013
Quote:
Originally Posted by dhruvmohan
I am new to awk scripting.

I want to do a field by word (field) comparison of two files File1.txt and File2.txt.

The files contain a list of | (pipe) separated field.

**File 1:
-------------------
Code:
aaa|bbb|ccc|eee|fff
lll|mmm|nnn|ooo|ppp
rrr|sss|ttt|uuu|vvv**

File 2:
-------------------
Code:
aaa|bbb|ccc|eee|fff
rrr|sss|ttt|uuu|vvv
rrr|sss|ttt|uuu|uuu

We compare the same line no. in both the files.
Fields in Line 1 of both file match.

In Line 2 all the fields (lll, mmm, nnn, ooo, ppp) donot not match with all fields (rrr, sss, ttt, uuu, vvv) in line 2 of File 2. Similarly the 5th field of 3rd line in both the files donot match.

Hence Line no. 2 and Line no. 3 should get echoed by bash.

Both files will follow an order.
Is this a homework item? It seems like a strange set of requirements for any non-classroom project.

You haven't shown the output that you want from the input given above, but it seems that you are asking for entire lines from File 1 or entire lines from File 2 to be written to standard output if any field is different. If that is the case, why should we compare fields instead of just comparing lines? Comparing lines should get the same results, is easier to program, and probably run faster.

If you are using awk to compare files; why is it important that bash use the echo command to print the results instead of having awk print the results directly?
# 3  
Old 09-22-2013
Thanks. You gt my question absolutely correct.

But I want to compare field by field instead of line by line so in case a field does not match I can echo out that field in output.
Hope you understood.
# 4  
Old 09-22-2013
I repeat:
  1. Is this a homework assignment? If not what is the real world project that is driving this?
  2. Show us the output you want (using CODE tags) corresponding to the sample input files you provided.
  3. Explain why you require us to use the bash echo command to print the results that an awk script is going to calculate.
# 5  
Old 09-22-2013
Thanks Don for your quick response.
Please find my comments below:

1). Well this is not a homework assignment. It is part of my performance testing project for a investment bank in US. Actually I have got a .csv file as a output of one application run. Now I need to verify the contents of this .csv file against the expected data.


2). Output will look like this:

Code:
Line 2: lll,mmm,nnn,ooo,ppp
Line 3: vvv

3). By echo out the output I meant I wanted to print the output in a file/ $ prompt (through print statement in awk). I did not mean of literally using the "echo" command only.

Hope I have answered all your questions. Please revert back to me in case of any further questions.
# 6  
Old 09-22-2013
In "File 1" line 3 you have:
Code:
rrr|sss|ttt|uuu|vvv**

and in "File 2" line 3 you have:
Code:
rrr|sss|ttt|uuu|uuu

Why is the output showing the differences in line 3 supposed to be:
Code:
Line 3: vvv

instead of:
Code:
Line 3: vvv**

# 7  
Old 09-22-2013
If the asterisks are not part of file 1 you could try something like this:
Code:
awk '
{
  s=x
  for(i=1; i<=NF; i++)
    if (NR==FNR) {
      A[NR,$i]
    } 
    else if(!((FNR,$i) in A)) s=(s?s OFS:x) $i
  if(s)print "Line " FNR ": " s
}
' FS=\| OFS=, file2 file1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Compare two files line by line in bash

Hello All! Thanks for taking time out and helping. My issue is, I have two files that have file names in it. Now, i need to go through each line of both the files and when the file names are different, i need to rename the file. Below is the example: File1</ fil1ename1.txt filename2,txt... (2 Replies)
Discussion started by: svks1985
2 Replies

2. Shell Programming and Scripting

Search every line and compare fields

Hi All, I have situation where my file looks like this log.file. ID Start time IP SQL 1256152 05-Aug-15, 11:25:06 MST 10.54.20.33 Select * from TES 1004768 05-Aug-15, 11:25:06 MST 10.54.20.33 Select dummy 323323 05-Aug-15, 12:00:06 MST 10.15.20.77 ... (1 Reply)
Discussion started by: netdbaind
1 Replies

3. Shell Programming and Scripting

How to read file line by line and compare subset of 1st line with 2nd?

Hi all, I have a log file say Test.log that gets updated continuously and it has data in pipe separated format. A sample log file would look like: <date1>|<data1>|<url1>|<result1> <date2>|<data2>|<url2>|<result2> <date3>|<data3>|<url3>|<result3> <date4>|<data4>|<url4>|<result4> What I... (3 Replies)
Discussion started by: pat_pramod
3 Replies

4. Shell Programming and Scripting

Comparison of fields then increment a counter reading line by line in a file

Hi, i have a scenario were i should compare a few fields from each line then increment a variable based on that. Example file 989878|8999|Y|0|Y|N|V 989878|8999|Y|0|N|N|V 989878|8999|Y|2344|Y|N|V i have 3 conditions to check and increment a variable on every line condition 1 if ( $3... (4 Replies)
Discussion started by: selvankj
4 Replies

5. Shell Programming and Scripting

Perl how to compare two pdf files line by line

Hi Experts, Would really appreciate if anyone can guide me how to compare two pdf files line by line and report the difference to another file. (3 Replies)
Discussion started by: prasanth_babu
3 Replies

6. Shell Programming and Scripting

how to read the contents of two files line by line and compare the line by line?

Hi All, I'm trying to figure out which are the trusted-ips and which are not using a script file.. I have a file named 'ip-list.txt' which contains some ip addresses and another file named 'trusted-ip-list.txt' which also contains some ip addresses. I want to read a line from... (4 Replies)
Discussion started by: mjavalkar
4 Replies

7. Shell Programming and Scripting

How to compare two files line by line

Using diff to compare 2 files FILE1: AAA AAA AAA AAA AAA AAA FILE2: BBB BBB AAA AAA BBB BBB diff FILE1 FILE2 (3 Replies)
Discussion started by: yannsun
3 Replies

8. Shell Programming and Scripting

Compare two files line by line

Query: There are two files as below - /home/rgupta/input/file.txt.arch (source file) /home/rgupta/output/file.txt (destination file) Files details are attached for the reference. Scenario: File /home/rgupta/input/file.txt.arch picked up by an application xyz. Application xyz does... (2 Replies)
Discussion started by: ravigupta2u
2 Replies

9. Shell Programming and Scripting

Compare two text files line by line

Hi all I need help on comparing two texts files line by line and print the total number of lines matched. Any help is appreciated. Thanks in advance. (4 Replies)
Discussion started by: my_Perl
4 Replies

10. Shell Programming and Scripting

Compare multiple fields in file1 to file2 and print line and next line

Hello, I have two files that I need to compare and print out the line from file2 that has the first 6 fields matching the first 6 fields in file1. Complicating this are the following restrictions 1. file1 is only a few thousand lines at most and file2 is greater than 2 million 2. I need to... (7 Replies)
Discussion started by: gillesc_mac
7 Replies
Login or Register to Ask a Question