Visit Our UNIX and Linux User Community


Differencing between two files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Differencing between two files
# 1  
Old 10-10-2007
Differencing between two files

I have two files. One from last month and one from this month. I need to identify the differences between the files so I can process only the differences. I ran the following command:

diff -C 1 bndl1111980740907_sort.txt bndl1131363071007_sort.txt > difftest.txt

It gave me a "-" in the first field for records in month one that are not in month two, great. It also gave me a "+" in the first field for records in month two that were not in month one. I'm good with that. I get a "!" for the records that are different between the two months. The problem is that I am getting two records when month two's record is different than month one's record. I get one "!" for month one and one for month two (month one always appears first). I only want the record for month two.

Is there a way to do this with diff and still keep the "+" and "-" records? I want only one "!" from month two. If I can do it with diff, can I use awk to filter out the second "!"?

Here is an example of the diff file.
+ C00000000DC* DCNS7 00121030416 000000000000000000000000000000000000000000000000000N1N01 C
! B0000ATC5 E 005080170029 A000035000020000000000000000019000000000000000000000 1P07EC
! B0000ATC5 E 005080170029 A000035000020000000000000000019000000000000000000000 1P09EC
! B0000ATM5 E 005080170029 A000050000030000000000000000029000000000000000000000 1P19EC
! B0000ATM5 E 005080170029 A000050000030000000000000000029000000000000000000000 1P21EC
! B00015866 E 00121109015X A001310000785000000000000000746000000000000000000000 1P11 B
! B00015866 E 00121109015X A001310000785000000000000000746000000000000000000000 1P12 B
! B00079583 E 001140890101 B003645002185000000000000002076000000000000000000000 1P01 C
! B00079583 E 001140890101 B003645002185000000000000002076000000000000000000000 1P02 C
! B000ATC10 E 005080170029 A000045000025000000000000000024000000000000000000000 1P24EC

The field that contains the part number is in position 4-13. For example, I would only want the second "!" record for part_number "00015866 ".

Can anyone help me?
# 2  
Old 10-10-2007
I have a feeling that you are looking only for the part numbers and not the entire fields for the parts which are not in 1 file or the other

if so..use the diff and cut the partnumber and then do a uniq sort or uniq..

cat diff_file | cut -d" " -f2 | sort -u

or insted or sort use, uniq -d..whatevevr..In this way, you get the parts which u may want to reporcess..


cheers,
Devaraj Takhellambam
# 3  
Old 10-10-2007
No, I need the entire record but the "key" to the record is the part number.

Can I use awk or sed to get rid of the first "!" from month one?

I was able to run the following with success. I am posting this in case someone else runs across this issue.

awk -F" " '{if ($1 == "-") { print; prev = substr($2,2,10) } else { {if ($1 == "+") { print; prev = substr($2,2,10) } else { {if (substr($2,2,10) != prev) { prev = substr($2,2,10) } else { print; prev = substr($2,2,10) } } } } } }' bndl_diffu2_awk2_sort2.txt > bndl_final.txt

Last edited by PSermonte; 10-11-2007 at 10:04 AM..

Previous Thread | Next Thread
Test Your Knowledge in Computers #685
Difficulty: Medium
Reverse Polish notation (RPN), also known as Polish programming notation.
True or False?

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Automate splitting of files , scp files as each split completes and combine files on target server

i use the split command to split a one terabyte backup file into 10 chunks of 100 GB each. The files are split one after the other. While the files is being split, I will like to scp the files one after the other as soon as the previous one completes, from server A to Server B. Then on server B ,... (2 Replies)
Discussion started by: malaika
2 Replies

2. Shell Programming and Scripting

Shell script for field wise record count for different Files .csv files

Hi, Very good wishes to all! Please help to provide the shell script for generating the record counts in filed wise from the .csv file My question: Source file: Field1 Field2 Field3 abc 12f sLm 1234 hjd 12d Hyd 34 Chn My target file should generate the .csv file with the... (14 Replies)
Discussion started by: Kirands
14 Replies

3. Shell Programming and Scripting

Append string to all the files inside a directory excluding subdirectories and .zip files

Hii, Could someone help me to append string to the starting of all the filenames inside a directory but it should exclude .zip files and subdirectories. Eg. file1: test1.log file2: test2.log file3 test.zip After running the script file1: string_test1.log file2: string_test2.log file3:... (4 Replies)
Discussion started by: Ravi Kishore
4 Replies

4. Shell Programming and Scripting

How to create zip/gz/tar files for if the files are older than particular days in UNIX or Linux?

I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also? I found one command which is to create gz file for the... (4 Replies)
Discussion started by: Mallikgm
4 Replies

5. Shell Programming and Scripting

need a shell script to extract the files from source file and check whether those files existonserve

Hi, I am new to shell scripting.Please help me on this.I am using solaris 10 OS and shell i am using is # echo $0 -sh My requirement is i have source file say makefile.I need to extract files with extensions (.c |.cxx |.h |.hxx |.sc) from the makefile.after doing so i need to check whether... (13 Replies)
Discussion started by: muraliinfy04
13 Replies

6. UNIX for Dummies Questions & Answers

write a program in c in unix that display the files(includ sub-direc and files within) in a sorted

the sorting is based on name of file, file size modification time stamps o f file it should dislay the output in the following format "." and ".." enteries should be ignored please give some idea how to do it (1 Reply)
Discussion started by: pappu kumar jha
1 Replies

7. Shell Programming and Scripting

How to extract data from indexed files (ISAM files) maintained in an unix server.

Hi, Could someone please assist on a quick way of How to extract data from indexed files (ISAM files) maintained in an UNIX(AIX) server.The file data needs to be extracted in flat text file or CSV or excel format . Usually we have programs in microfocus COBOL to extract data, but would like... (2 Replies)
Discussion started by: devina
2 Replies

8. Shell Programming and Scripting

Compare 2 folders to find several missing files among huge amounts of files.

Hi, all: I've got two folders, say, "folder1" and "folder2". Under each, there are thousands of files. It's quite obvious that there are some files missing in each. I just would like to find them. I believe this can be done by "diff" command. However, if I change the above question a... (1 Reply)
Discussion started by: jiapei100
1 Replies

9. Shell Programming and Scripting

How to retrieve all the linked script files/ctl files/sql files?

Hi I am going to migrate our datawarehouse system from HP Tru 64 Unix to the Red Hat Linux. Inside the box, it is running around 40 cron jobs; inside each cron job, it is calling other shell script files, and the shell script files may again call other shell script files or ctl files(for... (1 Reply)
Discussion started by: franksubramania
1 Replies

Featured Tech Videos