Unique File Differences


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Unique File Differences
# 1  
Old 04-08-2010
Unique File Differences

I have the 2 files

File 1
ABC,1239800
BCED,890000
ABCKJK,66767

File 2
GUHJC,1239800
ABC,1239800
TYIO,5636

The thing is the no of values in file can exceed example ABC,1239800,4545465,AHHAH so i need to find those values in file 1 which do not match in File 2 so i should get ABCKJK,66767 and BCED,890000
# 2  
Old 04-08-2010
Try:
Code:
awk -F, 'NR==FNR{A[$1];A[$2];next} !($1 in A || $2 in A)' file2 file1

Code:
BCED,890000
ABCKJK,66767

# 3  
Old 04-08-2010
I have done this way

Code:
cut -f6,7 -d"|" file1  | sort -u > /tmp/FILE1_$$
cut -f6,7 -d"|" file2 | sort -u > /tmp/FILE2_$$"
comm -3 FILE2 FILE1 > /tmp/NEWFILE_$$

And does your method work in case the no of columns increase ?

Last edited by Franklin52; 04-08-2010 at 04:57 AM.. Reason: Placing correct code tags
# 4  
Old 04-08-2010
MySQL

Code:
[root@sistem1lnx tes1]# cat diff.sh
cat /dev/null > diffline.txt
while read line1
do
  var=0
  while read line2
  do
    if [ "$line1" = "$line2" ]
      then var=1
    fi
  done < file2
  if [ "$var" = "0" ]
    then echo $line1 >> diffline.txt
  fi
done < file1

[root@sistem1lnx tes1]# ./diff.sh

[root@sistem1lnx tes1]# more diff
diffline.txt  diff.sh

[root@sistem1lnx tes1]# more diffline.txt
BCED,890000
ABCKJK,66767

# 5  
Old 04-08-2010
Something like this:
Code:
 
comm -23 File1 File2

Gives you the data present only in File1.

Make sure the files are sorted before doing this.
# 6  
Old 04-08-2010
I tried with comm but i have a problem as i need to do a case sensitive search as well as non case sensitive how should i proceed ?
# 7  
Old 04-09-2010
Then,

I would suggest you to use the "diff" command. Something like this:
Code:
 
diff -i File1 File2 | grep "<" | sed 's/^< //g'

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh / AIX - Differences between lists to a text file

This seems pretty simple, but I cant figure it out. I get stumped on the simple things. I am running two commands 1) take a listing a directory of files, and filter out the doc_name (which is in a series of extracted files), and place it in a file. ls -l | awk '{print $9}' | grep... (5 Replies)
Discussion started by: jeffs42885
5 Replies

2. Shell Programming and Scripting

awk to find differences between two file

I am trying to find the differences between the two sorted, tab separated, attached files. Thank you :). In update2 there are 52,058 lines and in current2 there are 52,197 so 139 differences should result. However, awk 'FNR==NR{a;next}!($0 in a)' update2 current2 > out2comm -1 -3... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

4. UNIX for Dummies Questions & Answers

Compare and merging the differences in text file

Hi i have gone through some sdiff command it shows the differences side by side and its really awesome file 1: this tool is for checking the differ merging with flower pots documentation file 2: this t ool is for checking the differ mergin g with flower pots documentation ... (27 Replies)
Discussion started by: rakeshkumar
27 Replies

5. Shell Programming and Scripting

Differences between 2 Flat Files and process the differences

Hi Hope you are having a great weeknd !! I had a question and need your expertise for this : I have 2 files File1 & File2(of same structure) which I need to compare on some columns. I need to find the values which are there in File2 but not in File 1 and put the Differences in another file... (5 Replies)
Discussion started by: newbie_8398
5 Replies

6. Shell Programming and Scripting

Compare File Differences in different directories

Hello, I am new to scripting and have been trying to compare two different directories, but with all the same file names in each directory for file changes. I have been doing it in baby steps and have been doing pretty good, but I have hit a few snags. Test 1 and Test 2 work great, but my... (4 Replies)
Discussion started by: dmaday
4 Replies

7. Shell Programming and Scripting

Help with file differences

I have two huge files in the size of 1gb. They are produced by similar processes and the expected thing is that they should match in size and contents. I have produced both the files with the processes and they seem to be off only by few bytes. Size file name 1634502037 ... (2 Replies)
Discussion started by: dsravan
2 Replies

8. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

9. Shell Programming and Scripting

Comparing files columnwise and print the differences in third file

Hello Everybody!!!!!!!!! Request you to help me with the below mentioned issue: I have 2 files say, File 1: a|4|7 b|3|2 c|8|8 d|8|9 File 2: a|4|6 b|2|2 c|8|8 d|9|8 The third file(output file) should have: Data mismatch in row 1 column 3 Data mismatch in row 2 coumn 2 Data... (3 Replies)
Discussion started by: abhijeet1409
3 Replies

10. Shell Programming and Scripting

comparing file content differences

I need to write a script to find out if there are any .c files created/removed from the last time i monitored the files available. i first created a file to contain all the .c files available on the system. (ls *.c > file1) I created another file using the same command. I used the comm file1... (4 Replies)
Discussion started by: RianTan
4 Replies
Login or Register to Ask a Question