i have 50 files .data should be same in these 50 files , so my task is to find the difference. i need a logic , which finds difference between all files and print in output file with file name where it found that difference .
i tried below logic , but its not giving me what i want.
let me give example
i tried this logic:-
cat $TMPDIR/*lis* | sort | egrep -ve $EXCEPTIONS_LIST_GREP | uniq -u| while read line
do
echo "Unique value found : ${line} : in file : `grep -lx "${line}" $TMPDIR/CELL*`"
aa=`grep -lx "${line}" *list*`
echo "$aa"
done >>$TMPDIR/ai see below data in 6 files which have differ by 1 file :-
echo from script :-
+ echo Unique value found : 2424480061 8303736 pbin/lib/libsyncsort.so : in file : /tmp/check_sync_20120207_231421_PID_29642/CELLTADM_LIST_2_celltadm@fra-siapp01_list
But we see three type of files in (1st column differes) , 1st-1213068711 , 2nd -2424480061 , 3rd-330122325
so it should give all these 3 difference.
actually i am comparing one directory structure on multiple servers, i copied all files name to different files (server name wise), now remaining thing to get the difference , on which server ,what file is different
Dear deepakiniimt, I'm a little confused, so please forgive me if I have missed the point. Perhaps putting code tags in your message would help clarity. It seems you may have chopped a bit out too, or i just don't "get it"
You say you have a bunch of files that you expect to be the same apart from a few differences. Could you consider the logic that if one file is the standard, then you could diff each one against it? Something like:-
Would that thinking suffice?
If you are looking for records that are unique in your list of files, then perhaps you should think about:
Find all unique records and capture to a file
Search for those records specifically (maybe edit the file first)
You could:-
Depending on your version of unix, you may need fgrep instead.
If this matches too many lines, then it may be that the lines are sometimes part of larger lines, eg:-
In this case, the last record would match the other three, along with any other record in any file that had the simple string "Hello". You could edit the file of unique records to make them all start with a carat symbol ^ and end with a dollar $ to explicitly signify begginning and end of line and the two ends of your string. You code would grow to:-
It's a bit clumsy and perhaps a sed or an awk would be quicker, just depending how much data you have.
Does that meet your needs, or have I gone off track completely?
I hope that it helps,
Robin
Liverpool/Blackburn
UK
Thanks a lot for prompt reply,
let me explain a bit more:-
1. each file is having 50K lines , though size of file is not too much.
2. i have no standard file , so that i cant compaer with standard file
3.to get unique lines first and then search again through these 50 lines is not fast , m not sure because no of lines are too much.
4. files are having 3 columns, actually this is 'cksum' value in files, so third column(file name), will be same.
well just now , i got some output ,looks meaningful :-
cat $TMPDIR/*list*|egrep -ve $EXCEPTIONS_LIST_GREP| sort | uniq -c|grep -vw ^"[ ]*5"|cut -d' ' -f 5-7|while read line
do
export aa=`grep -lx "${line}" $TMPDIR/*list*`
echo "Unique value found : ${line} : in file : $aa"
done >>$TMPDIR/aaaaaajust need to sort , i guess
output :-
Unique value found : 1213068711 8303736 pbin/lib/libsyncsort.so : in file : /tmp/check_sync_20120208_004328_PID_29046/CELLTADM_LIST_2_celltadm@avm-siapp01_list
/tmp/check_sync_20120208_004328_PID_29046/CELLTADM_LIST_2_celltadm@rpa-siapp01_list
/tmp/check_sync_20120208_004328_PID_29046/CELLTADM_LIST_2_celltadm@wlg-siapp01_list
Unique value found : 330122325 8303736 pbin/lib/libsyncsort.so : in file : /tmp/check_sync_20120208_004328_PID_29046/CELLTADM_LIST_2_celltadm@bat-siapp01_list
/tmp/check_sync_20120208_004328_PID_29046/CELLTADM_LIST_2_celltadm@rpb-siapp01_list
Hi All,
Am trying script to compare 2 files and print the difference found from old file to new file on line by line basis on side by side display.
Basically line by line comparision and files may contain blank line as well
I know we have compare/diff commands but i don't how to make... (10 Replies)
Hi experts,
I am trying to compare two text files and output the difference to another file.
I'm not strictly looking for differences in text but additional text at the end of one file that isn't in another, so basically comparing the file 2 against file 1 and printing any additional text to... (9 Replies)
It seems like a common task, but I haven't been able to find the solution.
vitallog.txt
1310,John,Hancock
13211,Steven,Mills
122,Jane,Doe
138,Thoms,Doe
1500,Micheal,May
vitalinfo.txt
12122,Jane,Thomas
122,Janes,Does
123,Paul,Kite
**OUTPUT**
vitalfiltered.txt
12122,Jane,Thomas... (2 Replies)
Hi there,
I'm sure this question has been asked many times but I can't find any posts with information.
How can I check the differences between say lines 20 - 200 in file1 and lines 420 - 600 in file2?
Thanks in advance for any help!
js (2 Replies)
I have two files like this:
#FILE 1
ABCD 4322 26485
JMTJ 5311 97248
XMPJ 4321 58978
#FILE 2
ABCD 4321 26485
JMTJ 5311 97248
XMPJ 4321 68978
What to do: Compare the two files and find those lines that doesn't match. And have a new file like this:
#FILE 3
"from file 1"
ABCD 4322 26485... (11 Replies)
I have to compare two files for any differences, then output the lab and question number for any differences. This is what I currently have:
diff lab2.txt lab2answer.txt > lab2compare.txt
Though the output doesn't have to be sent to a .txt (or any sort of log), I found that easier, at least... (2 Replies)
Hi,
I want to write a script which will compare the 1st column of both the files and will give the difference.
e.g:-
my 1st file contains:
89 /usr
52 /usr/local
36 /tmp
92 /opt
96 /home
27 /etc/opt/EMCom
1 ... (3 Replies)
Hi,
compare two .dat files and difference will be moved into separate file.if anybody having code for this please send asap. using diff command, i don't know how to write shell programming.
and my first file is like this including Header and trailer
10Ç20060323Ç01(Header)
01ÇIÇbabuÇ3000
01ÇIÇbaluÇ4000... (1 Reply)