Shell Programming and Scripting

{if(b[i]-1&&a[i]!=b[i]){print i";\t\t"b[i]}else{print "NEW:"i";\t\t"b[i]}

# cat old*
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:841068,SHELF:8,SLOT:4,02:00:03,Wed Dec 28 2011
NE:593848,SHELF:3,SLOT:1,02:30:09,Wed Dec 28 2011
NE:593848,SHELF:3,SLOT:1,02:30:09,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:841068,SHELF:8,SLOT:4,02:00:03,Wed Dec 28 2011
NE:593848,SHELF:3,SLOT:1,02:30:09,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011

# cat new
NE:888888,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:111111,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:221726,SHELF:8,SLOT:1,01:00:02,Wed Dec 28 2011
NE:888888,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:801048,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011
NE:841068,SHELF:8,SLOT:4,02:00:03,Wed Dec 28 2011
NE:593848,SHELF:3,SLOT:1,02:30:09,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011
NE:801048,SHELF:6,SLOT:2,04:00:01,Wed Dec 28 2011

# nawk -F, 'NR==FNR{a[$1FS$2FS$3]++;next}{b[$1FS$2FS$3]++}
END{for(j in b){bx++};for(i in a){for(j in b){if(i==j){cc=b[j]}else{cn++}
if(cn==bx){print "NEW->",i";\t\t",a[i];}w=0};if(w=!0&&cn<bx){;print i";\t\t",cc};w=1;cn=0}}
' new old*
NE:801048,SHELF:3,SLOT:1;                5
NE:593848,SHELF:3,SLOT:1;                3
NEW-> NE:888888,SHELF:3,SLOT:1;          2
NE:801048,SHELF:6,SLOT:2;                6
NEW-> NE:111111,SHELF:3,SLOT:1;          1
NE:841068,SHELF:8,SLOT:4;                2
NE:221726,SHELF:8,SLOT:1;                5

# nawk -F, 'NR==FNR{a[$1OFS$2OFS$3]++;next} a[$1OFS$2OFS$3]{b[$1OFS$2OFS$3]++}
> END{for(i in b){if(b[i]-1&&a[i]!=b[i]){print i";\t\t"b[i]}else{print "NEW:"i";\t\t"b[i]} } }' OFS=, new old*
NE:801048,SHELF:3,SLOT:1;               5
NE:593848,SHELF:3,SLOT:1;               3
NE:801048,SHELF:6,SLOT:2;               6
NE:841068,SHELF:8,SLOT:4;               2
NE:221726,SHELF:8,SLOT:1;               5

'NR==FNR{a[$1FS$2FS$3]++;next} ## execute this until the NR equal to FNR
so NR means  number of input records and it will be increase as long as tha read new records from all input files..
FNR means current record number in the current file so FNR holds the number of record for each new file as separately.
when the awk has started to execute for input files(from stdinput or pipe), NR and FNR equals 1
and both of them increases synchronous while started to read a new file..
when a new file is read from awk then NR will continue to increase but FNR reset to zero for every new file at each time.
so NR and FNR is same while a new file processing and so NR and FNR is eqaul for first file..

NR==FNR{a[$1FS$2FS$3]++;next} ## process when NR==FNR (so execute for first file which read ) and then
for example our line is "NE:888888,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011"
a[$1FS$2FS$3]++  ## $1 "NE:888888" FS=, and $2 "SHELF:3" and FS=, $3 "SLOT:1" and assing to array(a) 
a[NE:888888,SHELF:3,SLOT:1] --> our first index (is a string) array(a) holds our indexes (it is an associative array)
read goes on new records from all first file..
for exa read let's same record so new line is "NE:888888,SHELF:3,SLOT:1,01:30:02,Wed Dec 28 2011" ( same the above)
a[$1FS$2FS$3]++  ## 
increments the count for our index (NE:888888,SHELF:3,SLOT:1) so "print a[$1FS$2FS$3]" gives the count for this lines 
finally all lines accumulated in array indexes with counts

next ## go to next record(line). hereby awk is forced to read the next record
NR==FNR{....;next} ## so read all lines in first file.
When it comes to the end of first file so awk process next file(s)

{b[$1FS$2FS$3]++} ## in the same way accumulates all lines from other file(s)

END ## after all the files has been read then execute this code that in the END{...}
{for(j in b){bx++}; ## we find array length and assing to "bx" value (bx increase while read an index val in b array)

for(i in a){  ## read indexes in a array (i --> index ,, a --> array name)
for(j in b){  ## same...
if(i==j){cc=b[j]} ## if "NE:888888,SHELF:3,SLOT:1" equals to "NE:221726,SHELF:8,SLOT:1" (for exa) then "cc=b[j]" [index count]

else{cn++} ## else increase value the "cn" for how many indexes have for not equal to "NE:888888,SHELF:3,SLOT:1"
## so this comparing will continue while "for-loop" for b array indexes [ remember b array indexes come from old files not first]

if(cn==bx){print "NEW->",i";\t\t",a[i];}w=0} ## if cn equals to bx (so it's means that didnt find any matches between new file and other files indexes)
## set w to zero (for do not write as OLD value)
if then write as NEW value and a[i] (count for this index val)

if(w=!0&&cn<bx){;print i";\t\t",cc};w=1;cn=0}} ## if w val is non-zero and cn smaller than the bx val
## if w is differ from zero it means NEW value has not been found already (above) 
## and bx(length of b array is greater than cn (so there is a matching record)
## (remember these(i) values holds from new file indexes)
..actually it could have been as "-1" ) ## i modified like way for my code for this
## then can write as OLD value
## so write as OLD value and cc (with count for this(j) index which from other files (so b array))
' new old*  ## new file and old files

# nawk -F, 'NR==FNR{a[$1FS$2FS$3]++;next}{b[$1FS$2FS$3]++}
END{for(j in b){bx++};for(i in a){for(j in b){if(i==j){cc=b[j]}else{cn++}
if(cn==bx){print "NEW->",i";\t\t",a[i];}w=0};if(w=!0&&cn==bx-1){;print i";\t\t",cc};w=1;cn=0}}
' new old*

if(b[i]-1&&a[i]!=b[i])

# awk -F, 'NR==FNR{a[$1FS$2FS$3]++;next}{b[$1FS$2FS$3]++}
END{for(i in b){if(a[i]==b[i]){print "NEW->",i";\t\t",a[i]}else{print i";\t\t",b[i]}}}' new old* new

## if "i indexed element of b array's count" (other files and new records)
## and "i indexed element of a array's count" is equal (only new records)
## is equal that is to say -> there is only in new file and not in the old files
## because if there was a same record in the old files 
## then the record of count's would not be same and more than the a[i] 's count value..
## so it is a NEW line

if(b[i]-1&&a[i]!=b[i]) 
if(b[i]-1)
## if ["element that i indexed of b array's count - 1"] has a value
## so there must be at least one record (i guess be sure about read successfully the records )
a[i]!=b[i]
## if "i indexed element of b array's count" and "i indexed element of a array's count" is not equal
## so trying to be sure that is there a record in the other files?
## if not equal then there is a record in the other files
## so it is a OLD line