Diff two files with threshold value


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Diff two files with threshold value
# 8  
Old 09-16-2011
Try next 'Perl' script:
Code:
$ cat file1
abc|7000|jhon|2.3
xyz|9000|sam|6.7
pqr|8000|kapi|4.6
lmn|3000|kapi|4.6
$ cat file2
abc|7000|jhon|2.3
xyz|9000|sam|6.7
pqr|8000|kapi|4.5
lmn|3000|kapi|4.1
$ cat script.pl
use warnings;
use strict;

@ARGV == 2 or die qq[Usage: perl $0 file1 file2\n];

my $num_file = 1;
my %line;

while ( <> ) {
        chomp;
        my @f = split /\|/;

        if ( $num_file == 1 ) {
                $line{ "@f[0..$#f-1]" } = $f[-1];
                next;
        }

        if ( $num_file == 2 ) {
                if ( exists $line{ "@f[0..$#f-1]" } and abs( $line{ "@f[0..$#f-1]" } - $f[-1] ) > 0.1 ) {
                        printf "> %s\n< %s\n", 
                                join( "|", @f[0..$#f-1], $line{ "@f[0..$#f-1]" } ),
                                join( "|", @f[0..$#f-1], $f[-1] );
                }
        }
} continue {
        ++$num_file if eof;
}
$ perl script.pl
Usage: perl script.pl file1 file2
$ perl script.pl file1 file2
> lmn|3000|kapi|4.6
< lmn|3000|kapi|4.1

Regards,
Birei
# 9  
Old 09-16-2011
It work perfectly for forth column

But it don't catch the difference in 1 to 3 column

for example
file1
pqr|7000|jhon|2.3
xyz|9000|sam|6.7
pqr|8000|kapi|4.5
lmn|3000|kapi|4.6

file 2

pqr|7000|jhon|2.3
xyz|12000|sam|6.7
pqr|8000|kapi|4.5
lmn|3000|kapi|4.1

if the the second row second column is different it don't work
file1
xyz|9000|sam|6.7

file2
xyz|12000|sam|6.7
# 10  
Old 09-16-2011
Seven posts to try to make clear what you are trying to get, and still confused. I think we are not understanding each other, and may be my problem. I will quote:

You said:
Quote:
2) I have to sort the file on the second and third column
To question of spynappels:
Quote:
do you want output if the file occurs in only 1 file, or only if it appears in both files and has a difference of more than 0.1 in column 3?
your answer is:
Quote:
yes
only if it appears in both files
The program compares the first three fields of each line in both files. Only if they are the same compares the fourth column. In your last example the second field is different (9000 < 12000). For the program those lines are different so there is nothing to compare and nothing to send to output.

You post different input with each message. Please, try to help me a little and tell what 'same line in both files' means to you. Say what to do when field1, or field2, or field3 is different in both files. Say what to do if there is a line in one file that doesn't exists in the other one (all fields different) and post an example of input file with all cases and exactly the expected output.

Regards,
Birei.
# 11  
Old 09-16-2011
Birei sorry for not being clear,

the script work fine.
Perl is new to me so was not able to translate the logic.
but this script works fine for me Smilie

Thanks Birei for all your effort
# 12  
Old 09-16-2011
It does not matter.

You can safely indicate if the program doesn't work well, but understand that sometimes it is inconvenient to change the code because requirements are not clear from the beginning.

Regards,
Birei
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Diff 3 files, but diff only their 2nd column

Guys i have 3 files, but i want to compare and diff only the 2nd column path=`/home/whois/doms` for i in `cat domain.tx` do whois $i| sed -n '/Registry Registrant ID:/,/Registrant Email:/p' > $path/$i.registrant whois $i| sed -n '/Registry Admin ID:/,/Admin Email:/p' > $path/$i.admin... (10 Replies)
Discussion started by: kenshinhimura
10 Replies

2. Shell Programming and Scripting

Script for deleting files and directories when the file system reaches the threshold

Hi Can someone assist in writing a script. I have a filesystem named /sybase in my aix lpar. When this filesystem becomes 94% full all the files and directories under /sybase/logs should be deleted immediately. :confused: (7 Replies)
Discussion started by: newtoaixos
7 Replies

3. Shell Programming and Scripting

.procmailrc and uudeview (put attachments from diff senders to diff folders)

Moderator, please, delete this topic (1 Reply)
Discussion started by: optik77
1 Replies

4. Shell Programming and Scripting

Using Diff to Compare 2 files

Hi I've been trying various methods that I have found online with regards to comparing 2 files using the diff command. Nothing seems to work. The problem is that I'm not too familiar with the proper syntax. Can you please assist me. Here is my script: #!/bin/bash awk -F',' -v file1="$1"... (9 Replies)
Discussion started by: ladyAnne
9 Replies

5. Shell Programming and Scripting

diff bw two files

Hi All, I have two files which look as below File1 serial="1" name="abc" type="employee" field="IT" serial="2" name="cde" type="intern" field="Marketing" serial="3" name="pqr" type="contractor" field="IT" serial="4" name="xyz" type="employee" field="Sales" File2 serial="1"... (3 Replies)
Discussion started by: grajp002
3 Replies

6. Shell Programming and Scripting

diff of files

Hi, I have 2 files.I want to check if file1 is contained in file2. A.txt: ----- AAA BBB B.txt: ------ CCC AAA BBB DDD I want to check if A.txt is contained in B.txt. Can it be done using SED ? (12 Replies)
Discussion started by: giri_luck
12 Replies

7. Shell Programming and Scripting

Diff b/w 2 files

Hi Masters, I have two files named file1 and file2. Both the files contains the same contents with some difference in comments,space.But no content change. I tried to find the diff between the two files to make sure that contents are same. For that i tried diff -ibw file1 file2 But... (1 Reply)
Discussion started by: ecearund
1 Replies

8. Shell Programming and Scripting

Compare all files in a directory to a threshold value

Hi guys, I have the following, and would like to enhance it be be able to run it in the hard coded directory and compare each file in the directory with the expectedSizeHow would I go about doing this? Thanks, Bloke #!/bin/sh ] || { echo "Usage: watchSizes 400"; exit 0 ; } #Hammer: How... (1 Reply)
Discussion started by: Bloke
1 Replies

9. Shell Programming and Scripting

Find duplicates from multuple files with 2 diff types of files

I need to compare 2 diff type of files and find out the duplicate after comparing each types of files: Type 1 file name is like: file1.abc (the extension abc could any 3 characters but I can narrow it down or hardcode for 10/15 combinations). The other file is file1.bcd01abc (the extension... (2 Replies)
Discussion started by: ricky007
2 Replies

10. Shell Programming and Scripting

diff 2 files; output diff's to 3rd file

Hello, I want to compare two files. All records in file 2 that are not in file 1 should be output to file 3. For example: file 1 123 1234 123456 file 2 123 2345 23456 file 3 should have 2345 23456 I have looked at diff, bdiff, cmp, comm, diff3 without any luck! (2 Replies)
Discussion started by: blt123
2 Replies
Login or Register to Ask a Question