Shell Programming and Scripting

Hi Experts,

I have many CSV data files in the below format (Example) :-

Code:

Doc Number,Line Number,Condition Number
111,10,ABC
111,10,PQR
111,10,XYZ
222,20,DEF
222,20,EFG
222,20,HIJ
333,30,CCC
333,30,TCP

Now, for the above data i want to get the row count based on the Doc Number & Line Number combination (Excluding the Condition Number) i.e the program should consider only the Doc Number & Line Number as an unique combination & give me the record count, for the above example the record count should come as 3.
Can anyone please tell me which UNIX command can be used for this?

Thanks

Code:

 
$ nawk -F, '{a[$1","$2]++;next}END{for(i in a)print i"---->"a[i]}' input.txt
333,30---->2
222,20---->3
111,10---->3

Or is it this what you're looking for:

Code:

awk -F, '{a[$1$2]}END{for(i in a)s++;print s}' file

Hi Franklin,

Yes, this is what i was looking for, thanks a lot for your help , i will let you know in case i need any more help.

Hi Kamraj,

Thanks for your help as well

Perhaps also skip the header and use a comma (SUBSEP) in the index to prevent unintended blending of field1 and field2?

Code:

awk -F, 'NR>1{a[$1,$2]}END{for(i in a)s++;print s}' infile

Hi Experts,

Thanks
Just one more query, is it possible for me to run this awk command for a number of files in one go?
I mean, i have around 1000 files & i want to get the data count of all these 1000 files in one go rather then running the command 1000 times.

You can use multiple files, if you replace NR with FNR (if all these files have a header)..
But with 1000 files you'll run into system limits when you supply that many on the command line...

Do you only want the grand total or the total per file?