awk to find differences between two file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting awk to find differences between two file
# 1  
Old 03-19-2016
awk to find differences between two file

I am trying to find the differences between the two sorted, tab separated, attached files. Thank you Smilie.

In update2 there are 52,058 lines and in current2 there are 52,197 so 139 differences should result.

However,

Code:
awk 'FNR==NR{a[$0];next}!($0 in a)' update2 current2 > out2

Code:
comm -1 -3 update2 current2 > out2

just outputs the entire file not just the 139 different lines. Thank you Smilie.

edit: current2 was not tab seperated as I thought, sorry..... everything works.

Last edited by cmccabe; 03-19-2016 at 11:59 AM.. Reason: solved edit
# 2  
Old 03-19-2016
Quote:
Originally Posted by cmccabe
In update2 there are 52,058 lines and in current2 there are 52,197 so 139 differences should result.
139 is the difference in lines between both files:
Code:
echo $(( $(wc -l < current2) - $(wc -l < update2) ))
139

What you appear to be requesting is to find the unique lines between both files if space is not significant. What it is not clear is what to do if there are repeated lines in the same file.

Here's a Perl version that output only lines found once after processing every line from both files, space agnostic.
Code:
perl -anle '$d{(join "\t", @F)}++;END{for(keys %d){print if $d{$_}==1}}' current2  update2 > difference.output


Last edited by Aia; 03-19-2016 at 02:22 PM..
This User Gave Thanks to Aia For This Post:
# 3  
Old 03-22-2016
Thank you very much Smilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Sed/awk to tell differences between two lists

Greetings all, I have two output lists from a log that I am working with. Below are the examples. except, the lists are in the thousands. list1.out FEA1234 FEA4343 FEA3453 FEA3413 FEA34A3 FEA3433 .... list2.out FEA1235 (3 Replies)
Discussion started by: jeffs42885
3 Replies

2. Shell Programming and Scripting

How to do find differences between 2 XML Files?

Hello All, Requirement is to compare 2 XML files and see if there are any differences but from some of the providers We are receiving UTF-16 formatted XML file with no end of line as shown below. Excerpt of data file: ÿþ<^@?^@x^@m^@l^@ ^@v^@e^@r^@s^@i^@o^@n^@=^@"^@1^@.^@0^@"^@... (11 Replies)
Discussion started by: Ariean
11 Replies

3. Shell Programming and Scripting

Read column and find differences...

I have this file 427 A C A/C 12 436 G C G/C 12 445 C T C/T 12 447 A G A/G 9 451 T C T/C 5 456 A G A/G 12 493 G A G/A 12 I wanted to read the first column and find all other ids which are differences less than 10. 427 A C A/C 12 436 436 G C G/C 12 427,445... (7 Replies)
Discussion started by: empyrean
7 Replies

4. Shell Programming and Scripting

awk to compare 2nd and 3rd field and print the differences

need a one liner to compare 2nd and 3rd field and print values that are not matched in 2nd field Input col 2 col 3 1.1.1.1 11.11.11.11 8.8.8.8 0.0.0.0 3.3.3.3 2.2.2.2 7.7.7.7 3.3.3.3 5.5.5.5 1.1.1.1 4.4.4.4 6.6.6.6 9.9.9.9 output 7.7.7.7 ... (12 Replies)
Discussion started by: chidori
12 Replies

5. Shell Programming and Scripting

How to find a certain string in a file and replace it with a value from another file using sed/awk?

Hi Everyone, I am new to this forum and new to sed/awk programming too !! I need to find particular string in file1(text file) and replace it with a value from another text file(file2) the file2 has only one line and the value to be replaced with is in the second column. file 1: (assert (=... (21 Replies)
Discussion started by: paramad
21 Replies

6. Shell Programming and Scripting

Differences between 2 Flat Files and process the differences

Hi Hope you are having a great weeknd !! I had a question and need your expertise for this : I have 2 files File1 & File2(of same structure) which I need to compare on some columns. I need to find the values which are there in File2 but not in File 1 and put the Differences in another file... (5 Replies)
Discussion started by: newbie_8398
5 Replies

7. Shell Programming and Scripting

Unique File Differences

I have the 2 files File 1 ABC,1239800 BCED,890000 ABCKJK,66767 File 2 GUHJC,1239800 ABC,1239800 TYIO,5636 The thing is the no of values in file can exceed example ABC,1239800,4545465,AHHAH so i need to find those values in file 1 which do not match in File 2 so i should get... (7 Replies)
Discussion started by: dinjo_jo
7 Replies

8. HP-UX

Compare 2 systems to find any differences

Hi there, I have 2 machines running HP-UX. One off these controllers is able to send mail and the other cannot. I have looked at all the settings that I know and coannot find any differences. Is there a way to audit the 2 machinces by pulling all the settings then compare any differences? ... (2 Replies)
Discussion started by: lodey
2 Replies

9. Shell Programming and Scripting

Help with file differences

I have two huge files in the size of 1gb. They are produced by similar processes and the expected thing is that they should match in size and contents. I have produced both the files with the processes and they seem to be off only by few bytes. Size file name 1634502037 ... (2 Replies)
Discussion started by: dsravan
2 Replies

10. Shell Programming and Scripting

Differences in awk between UNIX flavours

Hi, I've been charged with the task of finding out whether the scripts which we use on our current DYNIX (Sequent) UNIX box will continue to run happily on our soon-to-be-installed Sun Solaris box. I'm fairly certain that they'll be OK, but I've heard mutterings about awk running differently... (2 Replies)
Discussion started by: pbritta
2 Replies
Login or Register to Ask a Question