Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

print the nuber of duplicates

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting print the nuber of duplicates
# 1  
Old 08-19-2009
print the nuber of duplicates
input

Code:
a
a
a
b
b
c

output

Code:
a 3
b 2
c 1

Could you guyz help me.
I tried uniq and wc -l to count uniq line
but kinda confused to print the no of duplicates
# 2  
Old 08-19-2009
use awk..
Code:
awk '{A[$1]++}END{for ( i in A){print i" "A[i]}}' filename

# 3  
Old 08-19-2009
Thanx working fine
# 4  
Old 08-20-2009
uniq with option -c can help you achieve this. If your file is unsorted, use sort before.
Code:
$ cat file
abc
def
abc
def
abc
ghi
abc
jkl

$ uniq -c file
      1 abc
      1 def
      1 abc
      1 def
      1 abc
      1 ghi
      1 abc
      1 jkl

$ sort file | uniq -c |  awk '{print $2" "$1}'
abc 4
def 2
ghi 1
jkl 1

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove duplicates

I have a file with the following format: fields seperated by "|" title1|something class|long...content1|keys title2|somhing class|log...content1|kes title1|sothing class|lon...content1|kes title3|shing cls|log...content1|ks I want to remove all duplicates with the same "title field"(the... (3 Replies)
Discussion started by: dtdt
3 Replies

2. UNIX for Dummies Questions & Answers

Filtering the duplicates

Hello, I want to filter all the duplicates of a record to one place. Sample input and output will give you better idea. I am new to unix. Can some one help me on this? Input: 7488 7389 chr1.fa chr1.fa 3546 9887 chr5.fa chr9.fa 7387 7898 chrX.fa chr3.fa 7488 7389 chr1.fa chr1.fa... (2 Replies)
Discussion started by: koneru_18
2 Replies

3. Shell Programming and Scripting

How to ID duplicates in a string

Hi guys, I am trying to identify the number of duplicate entries in a string inputed by the user. Here is a command I use: $ user_input="M T T" $echo "${user_input}" | awk '{for(i=0;i<=NF;i++) print $i }'| sort | uniq -d The above works fine for string with multiple letters. The problem is... (2 Replies)
Discussion started by: aoussenko
2 Replies

4. UNIX for Dummies Questions & Answers

print average of duplicates

Hi my inFile has 3 fields with duplicates in field1. I would like to print the average field2 and field3 for the duplicated field1. $cat inFile f1 f2 f3 A 7 2 B 4 2 B 2 3 C 6 5 D 15 2 D 5 3 D 10 4$cat outFile f1 f2 f3 A 7 2... (8 Replies)
Discussion started by: jdhahbi
8 Replies

5. Shell Programming and Scripting

Removing duplicates

Hi, I have a file in the below format., test test (10) to to (25) see see (45) and i need the output in the format of test 10 to 25 see 45 Some one help me? (6 Replies)
Discussion started by: imdadulla
6 Replies

6. UNIX for Dummies Questions & Answers

Duplicates

Hi, How to eliminate the duplicate values in unix? I have a excel file which contains duplicate values. Need to use this in a script. Thanks in advance. (3 Replies)
Discussion started by: venkatesht
3 Replies

7. Shell Programming and Scripting

need Shell script for Sort BASED ON FIRST FIELD and PRINT THE WHOLE FILE WITHOUT DUPLICATES

Can some one provide me a shell script. I have file with many columns and many rows. need to sort the first column and then remove the duplicates records if exists.. finally print the full data with first coulm as unique. Sort BASED ON FIRST FIELD and remove the duplicates if exists... (2 Replies)
Discussion started by: tuffEnuff
2 Replies

8. Shell Programming and Scripting

Search Duplicates, Print Line #

Masters, I have a text file in the following format. vrsonlviee RVEBAALSKE lyolzteglx UUOSIWMDLR pcybtapfee DKGFJBHBJO ozhrucfeau YQXATYMGJD cjwvjolrcv YDHALRYQTG mdukphspbc CQZRIOWEUB nbiqomzsgw DYSUBQSSPZ xovgvkneav HJFQQYBLAF boyyzdmzka BVTVUDHSCR vrsonlviee TGTKUCUYMA... (2 Replies)
Discussion started by: genehunter
2 Replies

9. Shell Programming and Scripting

Non Duplicates

I have input file like below. I00789524 0213 5212 D00789524 0213 5212 I00778787 2154 5412 The first two records are same(Duplicates) except I & D in the first character. I want non duplicates(ie. 3rd line) to be output. How can we get this . Can you help. Is there any single AWK or SED... (3 Replies)
Discussion started by: awk_beginner
3 Replies

10. HP-UX

getting duplicates

how to get duplicates in a file containing data in columns using command or scripting? (4 Replies)
Discussion started by: megh
4 Replies
Login or Register to Ask a Question