Visit Our UNIX and Linux User Community


Compare 2 text file with 1 column in each file and write mismatch data to 3rd file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare 2 text file with 1 column in each file and write mismatch data to 3rd file
# 1  
Old 10-24-2013
RedHat Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi,
I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file.
File1 - file2 = file3

Code:
wc -l file1.txt
58112
wc -l file2.txt
55260
 
head -5 file1.txt
101214200123
101214700300
101250030067
101214100500
109912312312
 
head -5 file2.txt
101250030067
101214200123
101214700333
109912312312
101214700300

I can sort the files.
What shall I do after that or any other way

Thanks,
Divya
# 2  
Old 10-24-2013
with awk

Code:
 $ awk 'NR==FNR{A[$1]++;next}{if(! A[$1]){print }else{A[$1]=0}}END{for(i in A){if(A[i]){print i}}}' file1 file2

101214700333
101214100500

This User Gave Thanks to pamu For This Post:
# 3  
Old 10-24-2013
You may try this also

Code:
$ awk 'FNR==NR{A[$1]++;next}{if(!($1 in  A))print;else delete A[$1]}END{for (i in A)print i}' file1 file2

101214700333
101214100500

This User Gave Thanks to Akshay Hegde For This Post:
# 4  
Old 10-28-2013
CAN YOU PLEASE EXPLAIN WHAT {A[$1]++" EXACTLY " DOES? IT IS CREATING AN ARRAY BUT NOT ASSIGNING ANYTHING. FURTHER INCREMENT SIGN IS NOT UNDERSTOOD BY ME. PLEASE HELP ME UNDERSTAND THIS.

Moderator's Comments:
Mod Comment No shouting in our forums. Do not type in all caps.
# 5  
Old 10-28-2013
You could use the comm command for this:
Code:
comm -23 file1.txt file2.txt

The files need to be sorted though
# 6  
Old 10-28-2013
Quote:
Originally Posted by paresh n doshi
CAN YOU PLEASE EXPLAIN WHAT {A[$1]++" EXACTLY " DOES? IT IS CREATING AN ARRAY BUT NOT ASSIGNING ANYTHING. FURTHER INCREMENT SIGN IS NOT UNDERSTOOD BY ME. PLEASE HELP ME UNDERSTAND THIS.
Please check below this may clear your doubts...


Code:
$ cat file

101250030067
101214200123
101214700333
109912312312
101214700300
101214700333
101214700333
109912312312

Code:
$ awk '{A[$1]++}END{for(i in A) print i,A[i]}' file

101214700333 3 # It has 3 occurrence in the file so A[i]=3
101214200123 1 # It has 1 occurrence in the file so A[i]=1
109912312312 2 # It has 2 occurrence in the file so A[i]=2
101214700300 1
101250030067 1

# 7  
Old 10-28-2013
Using the exist clause (x in A), it's indeed possible to define an array element without a value:
Code:
awk 'FNR==NR {A[$1]; next}
{if ($1 in  A) delete A[$1]; else print $1} END {for (i in A) print i}' file1 file2


Last edited by MadeInGermany; 10-28-2013 at 12:11 PM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #509
Difficulty: Medium
Most programming languages allow you to use whitespace characters (space, tab, etc.) in a variable's identifier.
True or False?

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Tee doesn't write all displayed data into text file

"Debain 9 - LXDE" I execute follow line in the bash terminal: /ts3/server/ts3server_startscript.sh start createinifile=1 | tee -a /ts3/server/key.txt The displayed output looks like follow: My key.txt file looks like follow: How can i save the whole displayed text in my file? Why does... (5 Replies)
Discussion started by: int3g3r
5 Replies

2. Shell Programming and Scripting

Column to row and position data in a text file

Hi everyone.. I have a list of values in a file... a, b, c, 1, 2, 3, aaaa, bbbbb, I am interested in converting this column to a row.. "text",aaaa, bbbb a,1 (7 Replies)
Discussion started by: manihi
7 Replies

3. Shell Programming and Scripting

Compare output of UNIX command and match data to text file

I am working on an outage script and I run a command from the command line which tells me the amount of generator failures in my market. The output of this command only gives me three digits to identify the site by. I have a master list of all sites in a separate file, call it list.txt. If my... (7 Replies)
Discussion started by: jbrass
7 Replies

4. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

5. Shell Programming and Scripting

Compare two files and write data to second file using awk

Hi Guys, I wanted to compare a delimited file and positional file, for a particular key files and if it matches then append the positional file with some data. Example: Delimited File -------------- Byer;Amy;NONE1;A5218257;E5218257 Byer;Amy;NONE1;A5218260;E5218260 Positional File... (3 Replies)
Discussion started by: Ajay Venkatesan
3 Replies

6. Shell Programming and Scripting

How to write text file data to excel using UNIX shell script?

Hi All, I have the requirement in unix shell script. I want to write the "ls -ltr" command out put to excel file as below. Input :text file data : drwxr-xr-x 5 root root 4096 Oct 2 12:26 drwxr-xr-x 2 apx aim 4096 Nov 29 18:40 drwxr-xr-x 5 root root 4096 Oct 2 12:26 drwxr-xr-x... (10 Replies)
Discussion started by: Balasankar
10 Replies

7. Shell Programming and Scripting

Count column data in a text file

I have a text file that has the following column data: 0.007 0.005 0.004 0.007 How do i output the total sum of the data above? (6 Replies)
Discussion started by: alegnagrp
6 Replies

8. Shell Programming and Scripting

Match list of strings in File A and compare with File B, C and write to a output file in CSV format

Hi Friends, I'm a great fan of this forum... it has helped me tone my skills in shell scripting. I have a challenge here, which I'm sure you guys would help me in achieving... File A has a list of job ids and I need to compare this with the File B (*.log) and File C (extend *.log) and copy... (6 Replies)
Discussion started by: asnandhakumar
6 Replies

9. Shell Programming and Scripting

using awk to substitute data in a column delimited text file

using awk to substitute data in a column delimited text file hello i would like to use awk to do the following calculation from the following snippet. input file C;2390 ;CV BOUILLOTTE 2L 2FACES NERVUREES ;1.00 ;3552612239004;13417 ;25 ;50 ; 12;50000 ; ; ... (3 Replies)
Discussion started by: iindie
3 Replies

10. UNIX for Dummies Questions & Answers

write data into a text file in bold format

Hi, can anyone help to write data into a text file in bold format and rollback to actual format. Thanks, Regards, Milton Y. (1 Reply)
Discussion started by: miltony
1 Replies

Featured Tech Videos