Shell Programming and Scripting

Code:

awk 'NR==FNR && ++a[$1] >1{b[$1]=1} NR!=FNR{if(b[$1]) {print >>"file1.txt"} else {print >> "file2.txt"}}' file1.txt file2.txt

That works absolutely perfectly! Thanks guys!

The only thing is I havent got a clue how it works, specifically how it knows to look in 'mainfile.txt' as it is not specified. There are a few other files being generated immediately before this line of code and are in the same directory so how does NR (number of records?)and FNR (all files?) know which file to look in (and it is definitely looking at mainfile.txt because i've checked).

Thanks again for all contributions and the solution

when NR==FNR, code read through the mainfile file to get which the repeat time of item ($1) is over 1(++a[$1]>1), and give them a tag (b[$1]=1), next, NR>FNR(or NR!=FNR), code read throught the mainfile again, for this time, based on your rule (b[$1]==1, file1; b[$1]==0, file2), generate two output files: file1 and file2. Does it make sense?

Hi Yinyuemi,

I'm afraid not. I understand the mathematics of how it decides if $1 is in more than once and if so what to do with it, and if not what to do with it.

it's just, at no point do i explicitly state "look in 'mainfile.txt' for this information"

there is a Master file. From this i strip some things out into another file. And again i strip some more data from this to create Mainfile.txt.

it seems no matter where i put this:

Code:

awk 'NR==FNR && ++a[$1] >1{b[$1]=1} NR!=FNR{if(b[$1]) {print >>"file1.txt"} else {print >> "file2.txt"}}' file1.txt file2.txt

no matter at which stage i place it as all the above files are created - at the beginning, in between them - it does the same thing and looks in 'mainfile.txt'. How?

I understand the high likelihood that this is just a case of me being a complete idiot and need something incredibly simple spelled out for me, but with only three months sporadic linux/bash programming under my belt that's not too surprising

Cheers again for your help.

Ian

really, I can't believe. tpyo in it?

Haha yeah it works, I already put in the '.txt' and the correct paths.

just dont know how it works...

---------- Post updated at 11:26 AM ---------- Previous update was at 09:34 AM ----------

OK,

remember how i said there is a Master file...

and from this i strip data out to produce another file...

and from this i strip more data out to get Mainfile.txt

and when i ran the array it somehow looked in Mainfile.txt and it all tallied up correctly.



well now i've tried using a different Master file. All the changes filter through all subsequent steps until it gets to the array part - and it is still outputting the same data as before .i.e it is still looking in the same place for it's data.

Is there any way i can explicitly state to look in mainfile.txt?

OK, scratch NR==FNR.

I tweaked jim's:-

Code:

awk ' {arr[$1]++; next} END{for (i in arr) {print arr[i], i }} ' mainfile.txt > t.tmp
awk ' FILENAME=="t.tmp" {arr[$2]=$1; next}
       FILENAME=="mainfile.txt" { if(arr[$1]>1) 
                                                   {print$0 >"file1.txt" }  
                                             else 
                                                  {print $0 >"file2.txt" } }' t.tmp mainfile.txt

this one does work perfectly. Thankyou to everyone who contributed

Ian

Moderator's Comments:

Please use code tags when posting data and code samples!