How to compare 2 files and create a result file with unmatched lines from first file.?


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to compare 2 files and create a result file with unmatched lines from first file.?
# 1  
Old 08-03-2015
How to compare 2 files and create a result file with unmatched lines from first file.?

HI,

I have 2 text files. file1 and file2.
Code:
file1.txt (There are no duplicates in this file)
1234
3232
4343
3435
6564
6767
1213

file2.txt 
1234,wq,wewe,qwqw
1234,as,dfdf,dfdf
4343,asas,sdds,dsds
6767,asas,fdfd,fdffd

I need to search each number in file1.txt in file2.txt's 1st field (fields are delimited by comma in file2.txt). If the match is not found in file2.txt then i need to put the unmatched numbers from file1.txt to file3.txt

Code:
output
file3.txt
3232
3435
6564
1213

can anyone give me a solution.
If possible using awk, since it is faster i believe.
# 2  
Old 08-03-2015
Code:
awk -F, 'FNR==NR {f2[$1];next} !($0 in f2)' file2.txt file1.txt

# 3  
Old 08-04-2015
Quote:
Originally Posted by vgersh99
Code:
awk -F, 'FNR==NR {f2[$1];next} !($0 in f2)' file2.txt file1.txt

Code:
Got the below error
awk: Input line cannot be longer than 3,000 bytes.

I guess the line size inside the 2nd file is more tha 3000 bytes.

Any other solution?
# 4  
Old 08-04-2015
what OS are you on? Do you have nawk? Do you have gawk?
# 5  
Old 08-04-2015
Quote:
Originally Posted by vgersh99
what OS are you on? Do you have nawk? Do you have gawk?
i am using HP-UX. there is no nawk or gawk commands on this.
Can i do it using any other command which takes less time?
currently i am using the below block
but i guess it will take a lot of time. There are more than 10000 lines in a file.
Code:
while read line
do
	search_count=`grep -c $line file2.txt`
	echo $grepped
	if [ $grepped -eq 0 ]; then
       echo $line >> unfetch.txt
	fi
done < file1.txt

# 6  
Old 08-04-2015
how about:
Code:
cut -d , -f1 file2 | awk 'FNR==NR {f2[$1];next} !($0 in f2)' - file1

# 7  
Old 08-04-2015
Please try this

Code:
sort file1.txt > tmp1
cut -d, -f1 file2.txt | sort | uniq > tmp
diff tmp tmp1 | grep '>' | sed -e 's/> //g' > file3.txt
rm -f tmp1 tmp

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 columns from the same file and print a value depending on the result

Hello Unix gurus, I have a file with this format (example values): label1 1 0 label2 1 0 label3 0.4 0.6 label4 0.5 0.5 label5 0.1 0.9 label6 0.9 0.1 in which: column 1 is a row label column 2 and 3 are values I would like to do a simple operation on this table and get the... (8 Replies)
Discussion started by: ksennin
8 Replies

2. Shell Programming and Scripting

Compare 2 files of csv file and match column data and create a new csv file of them

Hi, I am newbie in shell script. I need your help to solve my problem. Firstly, I have 2 files of csv and i want to compare of the contents then the output will be written in a new csv file. File1: SourceFile,DateTimeOriginal /home/intannf/foto/IMG_0713.JPG,2015:02:17 11:14:07... (8 Replies)
Discussion started by: refrain
8 Replies

3. Shell Programming and Scripting

Compare 2 file and write result

Hi friends...I have 2 files, file1.txt and reference.txt I could able to find difference using diff and following command awk 'NR == FNR { A=1; next } !A' reference.txt file1.txt above command listing data which is not in reference.txt 12.12 87 11.95 88 11.83 89 12.55 84... (12 Replies)
Discussion started by: Akshay Hegde
12 Replies

4. Shell Programming and Scripting

How to delete lines having unmatched delimeters in a file?

Hi, I have a delimited (|) file having millions of rows. Sometime the file comes with less number of delimiters than expected. In this scenario I want to delete the row having unmatched delimiters. Can anyone help me with the command to achieve this? Ex: File1.txt 8039339595113|JIMMY... (2 Replies)
Discussion started by: satyaatcgi
2 Replies

5. Shell Programming and Scripting

compare 2 files and return unique lines in each file (based on condition)

hi my problem is little complicated one. i have 2 files which appear like this file 1 abbsss:aa:22:34:as akl abc 1234 mkilll:as:ss:23:qs asc abc 0987 mlopii:cd:wq:24:as asd abc 7866 file2 lkoaa:as:24:32:sa alk abc 3245 lkmo:as:34:43:qs qsa abc 0987 kloia:ds:45:56:sa acq abc 7805 i... (5 Replies)
Discussion started by: anurupa777
5 Replies

6. Shell Programming and Scripting

Compare two files Field by field and output the result in another file

Hi Friends, Need Help. I have file1.txt as File1.txt |123|A|7267|Hyder|Cross|Sell|7801 |995|A|7051|2008|Lunar|New|Year|Promotion|7801 |996|A|7022|Q108|Targ|Prospect|&|SSCC|Savings|Promo|7801 |997|A|7182|Q1|Feb-Apr|08|Credit|ITA|PA|SBA|Campaign|7801 File2.txt... (7 Replies)
Discussion started by: i150371485
7 Replies

7. UNIX for Dummies Questions & Answers

Compare 2 files print the lines of file 2 that contain a string from file 1

Hello I am a new unix user, and I have a work related task to compare 2 files and print all of the lines in file 2 that contain a string from file 1 Note: the fields are in different columns in the files. I suspect the is a good use for awk? Thanks for your time & help File 1 123 232 W343... (6 Replies)
Discussion started by: KevinRidley
6 Replies

8. Shell Programming and Scripting

Read multiple log files and create output file and put the result

OS : Linux 2.6.9-67 - Red Hat Enterprise Linux ES release 4 Looking for a script that reads the following log files that gets generated everynight between 2 - 5am Master_App_20090717.log Master_App1_20090717.log Master_App2_20090717.log Master_App3_20090717.log... (2 Replies)
Discussion started by: aavam
2 Replies

9. Shell Programming and Scripting

Compare two csv files by two colums and create third file combining data from them.

I've got two large csv text table files with different number of columns each. I have to compare them based on first two columns and create resulting file that would in case of matched first two columns include all values from first one and all values (except first two colums) from second one. I... (5 Replies)
Discussion started by: agb2008
5 Replies

10. Shell Programming and Scripting

Outputting formatted Result log file from old 30000 lines result log<help required>

Well I have a 3000 lines result log file that contains all the machine data when it does the testing... It has 3 different section that i am intrsted in 1) starting with "20071126 11:11:11 Machine Header 1" 1000 lines... "End machine header 1" 2) starting with "20071126 12:12:12 Machine... (5 Replies)
Discussion started by: vikas.iet
5 Replies
Login or Register to Ask a Question