Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

How to count dup records in file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting How to count dup records in file?
# 1  
Old 10-21-2013
How to count dup records in file?
Hi Gurus,

I need to count the duplicate records in file
file
Code:
abc
abc
def
ghi
ghi
jkl

I want to get below result:
Code:
abc ,2
abc, 2
def ,1
ghi ,2
ghi, 2
jkl ,1

or
Code:
abc ,2 
def ,1
ghi,2
jkl,1

Thanks in advance
# 2  
Old 10-21-2013
Code:
cat <file> | uniq -c | sort -nr

This User Gave Thanks to port43 For This Post:
ken6503
# 3  
Old 10-21-2013
Your example file is sorted. If this is true, and you don't mind the output format
Code:
uniq -c file

awk can do the same and format the output
Code:
awk '(NR>1 && $0!=prev) {print prev,c; c=0} {c++; prev=$0} END {if (NR>1) print prev,c}' file

Instead of prev,c you can do prev", "c.
This User Gave Thanks to MadeInGermany For This Post:
ken6503
# 4  
Old 10-21-2013
try this

cat infile | awk " {rec[$1]=$1;c[$1]++} END { for (a in rec) print rec[a],c[a]} "
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Count records in a block

Hi, We have a file that has a structure like this: H10 1 2 3 D10 1 D20 1 2 3 D20 3 4 5 D20 4 5 6 D10 2 D20 1 2 3 D20 3 4 5 D20 4 5 6 S10 10 H10 1 2 3 D10 1 D20 1 2 3 D20 3 4 5 D20 4 5 6 D10 2 (2 Replies)
Discussion started by: jerome_rajan
2 Replies

2. Shell Programming and Scripting

Script to count particular type of records

Hi, I have a huge file containing thousands of records which are of following pattern: TYPE1 { originNodeType : "IVR" originHostName : "AAIVR" originTransactionID : "01310559" originTimeStamp : "20110620192440+0530" hostName : "hhhh" voucher : '0'D rProfileID : "ZZZZ" Before {... (5 Replies)
Discussion started by: madhukar1anand
5 Replies

3. Shell Programming and Scripting

Awk to Count Records with not null

Hi, I have a pipe seperated file I want to write a code to display count of lines that have 20th field not null. nawk -F"|" '{if ($20!="") print NR,$20}' xyz..txt This displays records with 20th field also null. I would like output as: (4 Replies)
Discussion started by: pinnacle
4 Replies

4. Shell Programming and Scripting

using awk to count no of records based on conditions

Hi I am having files with date and time stamp as the folder names like 200906051400,200906051500,200906051600 .....hence everyday 24 files will be generated i need to do certain things on this 24 files daily file contains the data like 200906050016370 0 1244141195225298lessrv3 ... (13 Replies)
Discussion started by: aemunathan
13 Replies

5. Shell Programming and Scripting

Record count based on a keyword in the records

Hi, Am having files with many records, i need to count and display the number of records based on the keyword in one of the column of the records. for e.g THE FILE CONTAINS TWO RECORDS LIKE. 200903031143150 0 1236060795054357lessrv1 BSNLSERVICE1 BSNLSERVICE1 ... (4 Replies)
Discussion started by: aemunathan
4 Replies

6. Shell Programming and Scripting

count characters in specific records

I have a text file which represents a http flow like this: HTTP/1.1 200 OK Date: Fri, 23 Jan 2009 17:16:24 GMT Server: Apache Last-Modified: Fri, 23 Jan 2009 17:08:03 GMT Accept-Ranges: bytes Cache-Control: max-age=540 Expires: Fri, 23 Jan 2009 17:21:31 GMT Vary: Accept-Encoding ... (1 Reply)
Discussion started by: littleboyblu
1 Replies

7. UNIX for Dummies Questions & Answers

Count records in a zip file

Hello, I searched the forums on the keywords in the title I used above, but I did not find the answer: Is it possible to count records in a .zip file on an AIX machine if i don't have pkunzip installed? From all the research I'm reading in google and the reading of pkunzip in Unix.com,... (3 Replies)
Discussion started by: tekster757
3 Replies

8. Shell Programming and Scripting

grep all records in a file and get a word count -perl

Hi, I have a file .. file.txt .. i need to get a total record count in the files into a $variable.. im using perl script thanks (4 Replies)
Discussion started by: meghana
4 Replies

9. Shell Programming and Scripting

To find the count of records from tables present inside a file.

hi gurus, I am having a file containing a list of tables.i want to find the count of records inside thes tables. for this i have to connect into database and i have to put the count for all the tables inside another file i used the following loop once all the tablenames are inside the file. ... (1 Reply)
Discussion started by: navojit dutta
1 Replies

10. Shell Programming and Scripting

Count No of Records in File without counting Header and Trailer Records

I have a flat file and need to count no of records in the file less the header and the trailer record. I would appreciate any and all asistance Thanks Hadi Lalani (2 Replies)
Discussion started by: guiguy
2 Replies
Login or Register to Ask a Question