Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Compare a file with all others then print off data

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Compare a file with all others then print off data
# 8  
Old 12-29-2011
When you run that piece of code, what is the result? Any error?

Regards,
Birei
This User Gave Thanks to birei For This Post:
llcooljatt
# 9  
Old 12-29-2011
can you provide me with the line i.e

Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1){print i";\t\t"b[i]}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111228.csv *.csv | sort

---------- Post updated at 06:11 PM ---------- Previous update was at 06:03 PM ----------

this prints everything as new and 1 multiple


Code:
NEW: NE:978303,SHELF:9,SLOT:1;          1
NEW: NE:978303,SHELF:9,SLOT:3;          1
NEW: NE:978303,SHELF:9,SLOT:4;          1
NEW: NE:984177,SHELF:2,SLOT:3;          1
NEW: NE:984177,SHELF:4,SLOT:2;          1
NEW: NE:984966,SHELF:3,SLOT:1;          1
NEW: NE:984966,SHELF:3,SLOT:3;          1
NEW: NE:985360,SHELF:2,SLOT:1;          1
NEW: NE:985360,SHELF:2,SLOT:4;          1
NEW: NE:985360,SHELF:3,SLOT:3;          1
NEW: NE:985360,SHELF:3,SLOT:5;          1
NEW: NE:985360,SHELF:4,SLOT:3;          1
NEW: NE:986089,SHELF:2,SLOT:4;          1
NEW: NE:986089,SHELF:3,SLOT:2;          1
NEW: NE:986089,SHELF:5,SLOT:3;          1
NEW: NE:986089,SHELF:6,SLOT:1;          1
NEW: NE:986089,SHELF:6,SLOT:5;          1
NEW: NE:990237,SHELF:2,SLOT:4;          1
NEW: NE:990237,SHELF:4,SLOT:3;          1
NEW: NE:990237,SHELF:7,SLOT:6;          1
NEW: NE:992658,SHELF:2,SLOT:3;          1
NEW: NE:992658,SHELF:2,SLOT:5;          1
NEW: NE:992658,SHELF:4,SLOT:2;          1
NEW: NE:992658,SHELF:5,SLOT:3;          1
NEW: NE:992711,SHELF:1,SLOT:4;          1
NEW: NE:992711,SHELF:2,SLOT:1;          1
NEW: NE:993180,SHELF:2,SLOT:4;          1
NEW: NE:993180,SHELF:3,SLOT:5;          1
NEW: NE:993180,SHELF:5,SLOT:4;          1
NEW: NE:993180,SHELF:5,SLOT:6;          1
NEW: NE:993880,SHELF:1,SLOT:4;          1

# 10  
Old 12-29-2011
Quote:
Originally Posted by llcooljatt
can you provide me with the line i.e

Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1){print i";\t\t"b[i]}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111228.csv *.csv | sort

..........
...........
maybe you can try like this [ as far as i understand what you want ]
Code:
# awk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next}{b[$1OFS$2OFS$3]++}
END{for(i in a){for(j in b){if(i==j){cc=a[i]+b[j]}else{cn++};
if(cn==length(b)){print "NEW ENTRY FOUND -->",i,a[i];w=0}};if(w!=0){print i,cc};w=1;cn=0}}
' new old*

regards
ygemici
This User Gave Thanks to ygemici For This Post:
llcooljatt
# 11  
Old 12-30-2011
Hi I will check when I'm in work, can you check my previous posts, I had help from initial script from ahamed but just need it to do one more thing i.e increment NEW resets it has found i.e not look for -1 but give correct figure as NEW if the line is not in *.csv

---------- Post updated 30-12-11 at 07:21 AM ---------- Previous update was 29-12-11 at 09:22 PM ----------

#nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next}{b[$1OFS$2OFS$3]++}
> END{for(i in a){for(j in b){if(i==j){cc=a[i]+b[j]}else{cn++};
> if(cn==length(b)){print "NEW ENTRY FOUND -->",i,a[i];w=0}};if(w!=0){print i,cc};w=1;cn=0}}
> ' 20111228.csv *.csv
nawk: can't read value of b; it's an array name.
input record number 45, file 20111228.csv
source line number 3
# 12  
Old 12-30-2011
Quote:
Originally Posted by llcooljatt
Hi I will check when I'm in work, can you check my previous posts, I had help from initial script from ahamed but just need it to do one more thing i.e increment NEW resets it has found i.e not look for -1 but give correct figure as NEW if the line is not in *.csv

---------- Post updated 30-12-11 at 07:21 AM ---------- Previous update was 29-12-11 at 09:22 PM ----------

#nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next}{b[$1OFS$2OFS$3]++}
> END{for(i in a){for(j in b){if(i==j){cc=a[i]+b[j]}else{cn++};
> if(cn==length(b)){print "NEW ENTRY FOUND -->",i,a[i];w=0}};if(w!=0){print i,cc};w=1;cn=0}}
> ' 20111228.csv *.csv
nawk: can't read value of b; it's an array name.
input record number 45, file 20111228.csv
source line number 3
then try like this
Code:
# nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next}{b[$1OFS$2OFS$3]++}
END{for(j in b){bx++};for(i in a){for(j in b){if(i==j){cc=a[i]+b[j]}else{cn++}
if(cn==bx){print "NEW ENTRY FOUND -->",i,a[i];w=0}};if(w!=0){print i,cc};w=1;cn=0}}
' 20111228.csv *.csv

This User Gave Thanks to ygemici For This Post:
llcooljatt
# 13  
Old 12-30-2011
your script
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next}{b[$1OFS$2OFS$3]++}
> END{for(j in b){bx++};for(i in a){for(j in b){if(i==j){cc=a[i]+b[j]}else{cn++}
> if(cn==bx){print "NEW ENTRY FOUND -->",i,a[i];w=0}};if(w!=0){print i,cc};w=1;cn=0}}
> ' failed_lcss_reboots_20111229.csv *.csv
NE:565538 SHELF:10 SLOT:4 5
NE:851608 SHELF:4 SLOT:2 12
NE:504024 SHELF:4 SLOT:2 2
NE:571528 SHELF:1 SLOT:5 3
NE:236260 SHELF:15 SLOT:5 2
NE:220172 SHELF:3 SLOT:4 4
NE:239160 SHELF:11 SLOT:5 18
NE:221478 SHELF:3 SLOT:2 4
NE:565538 SHELF:7 SLOT:4 19
NE:883948 SHELF:10 SLOT:2 2
NE:564127 SHELF:8 SLOT:4 3
NE:573910 SHELF:5 SLOT:4 2
NE:602312 SHELF:3 SLOT:2 5
NE:972774 SHELF:5 SLOT:2 2
NE:993180 SHELF:2 SLOT:6 5
NE:978303 SHELF:3 SLOT:2 3
NE:221478 SHELF:10 SLOT:4 10
NE:571452 SHELF:11 SLOT:5 6
NE:903793 SHELF:3 SLOT:2 3
NE:239337 SHELF:8 SLOT:1 4
NE:852991 SHELF:11 SLOT:4 24
NE:222372 SHELF:5 SLOT:4 3
NE:267909 SHELF:4 SLOT:4 6
NE:858436 SHELF:6 SLOT:5 4
NE:220251 SHELF:7 SLOT:4 6
NE:509622 SHELF:12 SLOT:2 4
NE:238974 SHELF:12 SLOT:4 3
NE:236313 SHELF:16 SLOT:3 7
NE:851568 SHELF:5 SLOT:6 6
NE:554105 SHELF:5 SLOT:5 8
NE:221768 SHELF:4 SLOT:6 2
NE:571528 SHELF:12 SLOT:5 5
NE:828273 SHELF:2 SLOT:2 9
NE:575322 SHELF:8 SLOT:1 4
NE:565769 SHELF:5 SLOT:2 3
NE:597708 SHELF:11 SLOT:5 2
NE:509033 SHELF:2 SLOT:1 27
NE:220012 SHELF:5 SLOT:5 6
NE:220151 SHELF:6 SLOT:1 2
NE:829608 SHELF:1 SLOT:6 3


previous script


Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1){print i";\t\t"b[i]}else{print "NEW:"i";\t\t1"} } }' OFS=, 20111229.csv *.csv | sort
NE:220012,SHELF:5,SLOT:5;               3
NE:220172,SHELF:3,SLOT:4;               3
NE:220251,SHELF:7,SLOT:4;               5
NE:221478,SHELF:10,SLOT:4;              7
NE:221478,SHELF:3,SLOT:2;               3
NE:222372,SHELF:5,SLOT:4;               2
NE:236313,SHELF:16,SLOT:3;              4
NE:238974,SHELF:12,SLOT:4;              2
NE:239160,SHELF:11,SLOT:5;              17
NE:239337,SHELF:8,SLOT:1;               3
NE:267909,SHELF:4,SLOT:4;               5
NE:509033,SHELF:2,SLOT:1;               20
NE:509622,SHELF:12,SLOT:2;              3
NE:554105,SHELF:5,SLOT:5;               7
NE:564127,SHELF:8,SLOT:4;               2
NE:565538,SHELF:10,SLOT:4;              3
NE:565538,SHELF:7,SLOT:4;               17
NE:565769,SHELF:5,SLOT:2;               2
NE:571452,SHELF:11,SLOT:5;              5
NE:571528,SHELF:1,SLOT:5;               2
NE:571528,SHELF:12,SLOT:5;              4
NE:575322,SHELF:8,SLOT:1;               3
NE:602312,SHELF:3,SLOT:2;               3
NE:828273,SHELF:2,SLOT:2;               8
NE:829608,SHELF:1,SLOT:6;               2
NE:851568,SHELF:5,SLOT:6;               5
NE:851608,SHELF:4,SLOT:2;               11
NE:852991,SHELF:11,SLOT:4;              21
NE:858436,SHELF:6,SLOT:5;               3
NE:903793,SHELF:3,SLOT:2;               2
NE:978303,SHELF:3,SLOT:2;               2
NE:993180,SHELF:2,SLOT:6;               4
NEW:NE:220151,SHELF:6,SLOT:1;           1
NEW:NE:221768,SHELF:4,SLOT:6;           1
NEW:NE:223142,SHELF:11,SLOT:5;          1
NEW:NE:236260,SHELF:15,SLOT:5;          1
NEW:NE:504024,SHELF:4,SLOT:2;           1
NEW:NE:573910,SHELF:5,SLOT:4;           1
NEW:NE:597708,SHELF:11,SLOT:5;          1
NEW:NE:883948,SHELF:10,SLOT:2;          1
NEW:NE:972774,SHELF:5,SLOT:2;           1



your script seems to add an extra 1 to the figures, where as my old one is pretty much there only thing I need it to do is not look for -1 for new resets when comparing 20111229.csv with *.csv but give accurate reflection i.e instead of 1 this could be 10

i.e

NEW:NE:883948,SHELF:10,SLOT:2; 10

highlight as NEW on left
# 14  
Old 12-30-2011
Code:
nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
END{for(i in b){if(b[i]-1&&a[i]!=b[i]){print i";\t\t"b[i]}else{print "NEW:"i";\t\t"b[i]} } }' OFS=, 20111229.csv *.csv

--ahamed
This User Gave Thanks to ahamed101 For This Post:
llcooljatt
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Compare 2 columns from the same file and print a value depending on the result

Hello Unix gurus, I have a file with this format (example values): label1 1 0 label2 1 0 label3 0.4 0.6 label4 0.5 0.5 label5 0.1 0.9 label6 0.9 0.1 in which: column 1 is a row label column 2 and 3 are values I would like to do a simple operation on this table and get the... (8 Replies)
Discussion started by: ksennin
8 Replies

2. Shell Programming and Scripting

Compare & print content of file which is not matching

Hi All I want to compare 2 files using awk and get output of content which is not matching I have 2 files a.txt 123 456 780 143 b.txt A|B|C|167|D|E C|K|D|123|D|E A|B|D|789|G|F C|D|G|143|A|B Not matching line from b.txt O/P A|B|C|167|D|E A|B|D|789|G|F (3 Replies)
Discussion started by: aaysa123
3 Replies

3. Shell Programming and Scripting

Compare 2 text file with 1 column in each file and write mismatch data to 3rd file

Hi, I need to compare 2 text files with around 60000 rows and 1 column. I need to compare these and write the mismatch data to 3rd file. File1 - file2 = file3 wc -l file1.txt 58112 wc -l file2.txt 55260 head -5 file1.txt 101214200123 101214700300 101250030067 101214100500... (10 Replies)
Discussion started by: Divya Nochiyil
10 Replies

4. UNIX for Dummies Questions & Answers

Compare 2 files print the lines of file 2 that contain a string from file 1

Hello I am a new unix user, and I have a work related task to compare 2 files and print all of the lines in file 2 that contain a string from file 1 Note: the fields are in different columns in the files. I suspect the is a good use for awk? Thanks for your time & help File 1 123 232 W343... (6 Replies)
Discussion started by: KevinRidley
6 Replies

5. Shell Programming and Scripting

Compare and print out data only appear in file 1 problem

Below is the data content of file_1 and file_2: file_1 >sample_1 FKGJGPOPOPOQA ASDADWEEWERE ASDAWEWQWRW ASDASDASDASDD file_2 >sample_1 DRTOWPFPOPOQA ASDADWEEASDF ASDADRTYWRW ASDASDASDASDD I got try the following perl script. Unfortunately, it can't give my desired output result... (7 Replies)
Discussion started by: patrick87
7 Replies

6. Shell Programming and Scripting

Compare two file and print same line

i want to compare two file and print same line file1 12345 a 23456 a 45678 a 45679 a file2 23456 a 34567 a 45679 a output 23456 a 45679 a any one can help me? Thank you (7 Replies)
Discussion started by: bleach8578
7 Replies

7. Shell Programming and Scripting

compare two columns of different files and print the matching second file..

Hi, I have two tab separated files; file1: S.No ddi fi cu o/l t+ t- 1 0.5 0.6 o 0.1 0.2 2 0.2 0.3 l 0.3 0.4 3 0.5 0.8 l 0.1 0.6 ... (5 Replies)
Discussion started by: vasanth.vadalur
5 Replies

8. Shell Programming and Scripting

Compare selected columns from a file and print difference

I have learned file comparison from my previous post here. Then, it is comparing the whole line. Now, i have a new problem. I have two files with 3 columns separated with a "|". What i want to do is to compare the second and third column of file 1, and the second and third column of file 2. And... (4 Replies)
Discussion started by: kingpeejay
4 Replies

9. Shell Programming and Scripting

compare 2 file and print difference in the third file URG PLS

Hi I have two files in unix. I need to compare two files and print the differed lines in other file Eg file1 1111 2222 3333 file2 1111 2222 3333 4444 5555 newfile 4444 5555 Thanks In advance (3 Replies)
Discussion started by: evvander
3 Replies

10. UNIX for Dummies Questions & Answers

Compare Data in the same file

Dear Unix-Gurus, I'm trying to write a script to compare the data in a log file. Here's how my logfile will look like: 'List All A0 Data in Destination Server' A0567 A0678 A0789 List A0 Files in Source Server A0567 A0678 A0789 So if the file match in Source Server match Destination... (1 Reply)
Discussion started by: lweegp
1 Replies
Login or Register to Ask a Question