Comparing two unsorted files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Comparing two unsorted files
# 1  
Old 05-19-2011
Comparing two unsorted files

Hi Guys,

I'm a complete shell scripting newbie and need some help with comparing a file against a master file and outputting the results.

master.txt would look something like this:
Code:
000123
000345
000341
000927
000762
000235
000155
000452
000846
000623

file.txt would look like this:
Code:
000452
000341
000927

I need to compare file.txt against master.txt and output two files:

One file would be the numbers which are in both master.txt and file.txt
eg. same.txt
Code:
000452
000341
000927

And the other file would be the numbers which are in master.txt, but not in file.txt
eg. notsame.txt
Code:
000123
000345
000762
000235
000155
000846
000623

Any help would be greatly appreciated!
# 2  
Old 05-19-2011
Assuming that neither file is too large to pose an issue for sort or /tmp:

Code:
sort master.txt >/tmp/JNK$$.master
sort file.txt >/tmp/JNK$$.file
comm -12 /tmp/JNK$$.master /tmp/JNK$$.file >/tmp/in_both
comm -23 /tmp/JNK$$.master /tmp/JNK$$.file >/tmp/just_master
rm /tmp/JNK$$.*

Check the manual page for comm for description of the options; I've been using comm for years and have to read the man page nearly every time I need to use it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Comparing two files and list the difference with common first line content of both files

I have two file as given below which shows the ACL permissions of each file. I need to compare the source file with target file and list down the difference as specified below in required output. Can someone help me on this ? Source File ************* # file: /local/test_1 # owner: own #... (4 Replies)
Discussion started by: sarathy_a35
4 Replies

2. Shell Programming and Scripting

Compare two unsorted unequal files extracted from xml

I have two files for comparison which are extracts from set of xml files. file1 has: Comparing File: BRCSH1to320140224CC3.xml :: TZZZ:BR :: TAZZ:OUT UIZZ:0 :: ERAZ:1.000000 UIZZ:0 :: CTZZ:B UIZZ:0 :: CCAZ:MYR Comparing File: BRMY20140224CC18REG013SPFNSY13.xml :: TZZZ:BR :: TAZZ:INB... (1 Reply)
Discussion started by: vamsi gunda
1 Replies

3. Red Hat

Free() corrupted unsorted chunks

We are migrating Pro*C code from SOLARIS to LINUX-Redhat. While migrating we face memory de-allocation issue intermittently when accessing large volume of data. Below is the part of the code(since code is big I am putting the part of the code where the issue comes): ... (8 Replies)
Discussion started by: Karunx
8 Replies

4. Shell Programming and Scripting

Comparing two large unsorted csv files

Hi All, My requirement is to write a shell script to compare two large csv files. I've created sample files for explaining my problem i.e., a.csv and b.csv contents of files: ----------------- a.csv ------ Type,Memory (Kb),Location HD,Size (Mb),Serial # XT,640,D402,0,MG0010... (2 Replies)
Discussion started by: vasavi
2 Replies

5. Shell Programming and Scripting

Comparing files in a directory against an array of files

I hope I can explain this correctly. I am using Bash-4.2 for my shell. I have a group of file names held in an array. I want to compare the names in this array against the names of files currently present in a directory. If the file does not exist in the directory, that is not a problem.... (5 Replies)
Discussion started by: BudMan
5 Replies

6. UNIX for Advanced & Expert Users

How to find duplicates contents in a files by comparing other files?

Hi Guys , we have one directory ...in that directory all files will be set on each day.. files must have header ,contents ,footer.. i wants to compare the header,contents,footer ..if its same means display an error message as 'files contents same' (7 Replies)
Discussion started by: Venkatesh1
7 Replies

7. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

8. Shell Programming and Scripting

Need help comparing two files and deleting some things in those files!

So I have two files: File1 pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2 pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2 ref4948 1.1 treehouse.txt 1.6 ref8573 1.5 ref3284 1.4 ref5838... (24 Replies)
Discussion started by: linuxkid
24 Replies

9. Shell Programming and Scripting

comparing of two files

i have two file e.g cmp1 and cmp2 cmp1 has abcdef and cmp2 has abcdefghijkl cmp1 has: a b c d e f ========= cm2 has: a b c d e f g h (2 Replies)
Discussion started by: Aditya.Gurgaon
2 Replies

10. UNIX for Advanced & Expert Users

comparing shadow files with real files

Hi I need to compare shadow file sizes with their real file counterparts. If the shadow file size differs form the realfile size then it must send a mail. My problem is that our system has over 1600 shadowfiles in different directories, with different names. the only consistancy is the .sh file... (4 Replies)
Discussion started by: terrym
4 Replies
Login or Register to Ask a Question