Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Request to check remove duplicates but write before it

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Request to check remove duplicates but write before it
# 1  
Old 07-17-2012
Request to check remove duplicates but write before it
Hi alll

I have a file with following kind input

Quote:
1. X
1. Y
1. Z
2. X
2. B
2. C
2. D
2. Z

I want in output duplicates should not be there but there should be numbering mentioned before that like

Quote:
X. 1,2
Y. 1
Z. 1
B. 2
C. 2
D. 2
Z. 1,2
# 2  
Old 07-17-2012
Code:
awk -F. '{a[$2]=a[$2]" "$1}END{for(i in a)print i"."a[i]}' infile

# 3  
Old 07-18-2012
Request to check
Hi

Thanks for reply. I am not getting result using followin commnad is there any error?

Quote:
bash-3.2$ awk -F. '{a[$2]=a[$2]" "$1}END{for(i in a)print i"."a[i]}'Sara's data >* sara1>
actually the inputdata is not as numbered 1,2

it's like this:input is actually a long list like this

Quote:
CAD AEZ
CAD exy
CAD fryu
RA AEZ
RA ghdsj
RA fryu
BD fryu
BD jsfhdkl
BD AEZ
CD fsdgsdfg
CD AEZ
T1D sfgtsdfg
T1D fryu
T2D 465
T2D fryu
HT AEZ
HT sdfgsdg
so I wan output shuld contain

Quote:
AEZ CAD,RA,HT,CD
fryu T1D,T2D,RA,BD,CAD
and so on......
common entries shuld be mentioned in front of those .and duplicates shuld be removed.
# 4  
Old 07-18-2012
Code:
$awk 'BEGIN{i=0}{if(!a[$2]){b[i++]=$2;a[$2]=$2" "$1}else{a[$2]=a[$2]","$1}}END{for (j=0;j<i;j++){print a[b[j]]}}'  inputfile

# 5  
Old 07-18-2012
That's about the same question as in your other thread.
https://www.unix.com/shell-programmin...#post302673689

Please refrain from double posts which I consider your 2 posts having the same type of question.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

2. Shell Programming and Scripting

Kindly check:remove duplicates with similar data in front of it

Hi all, I have 2 files containing data like this: so if there is same entry repeated in the column like1,2,3,4 I have to check if there is different entries column like 2,4 but similar entries for duplicatein column 2 like1,3 the output shuld be like this for first file ... (5 Replies)
Discussion started by: manigrover
5 Replies

3. Shell Programming and Scripting

Request to check:remove duplicates and write sytematically

Hi all I have a file with following input It contains 5 columns gene name drug drug ID disease approved Now the same gene is repeated many times with different data in column2,3 ,4,5 I want to arrange dat in such a way that there shuld be one entry in the column(no... (2 Replies)
Discussion started by: manigrover
2 Replies

4. Shell Programming and Scripting

Request to check: compare two files , match same entries, write data before it

Hi all, I have 2 files:Column1 of first file has to be matched with column 3 of second file first file contain DATA like this in 2 columns one with gene name second with whether CAD,HT,RA T2Dor any one column 1 column2 ARFGEF2 CAD DDEF2 CAD PSCD3 CAD PSCD4 CAD CAMK1... (5 Replies)
Discussion started by: manigrover
5 Replies

5. Shell Programming and Scripting

Request to check: find common and write before it

Hi all, I have 2 big files with such kind of inputs File I File II I want the output file shuld contain Please let me knw scripting regarind this (1 Reply)
Discussion started by: manigrover
1 Replies

6. Shell Programming and Scripting

Request to check:Remove duplicates

Hi all I have a file with following kind of data I want to remove duplicates according to first column so that output contains Kindly let me scripting regading this. (4 Replies)
Discussion started by: manigrover
4 Replies

7. Shell Programming and Scripting

Request to check:remove entries more than once in different column

Hi I have a file 12m 345693460 12 1234 12 1234 34 345 34 345 And I want output fiel as 12m 345693460 12 1234 34 345 hw can it be done Thanks (1 Reply)
Discussion started by: manigrover
1 Replies

8. Shell Programming and Scripting

Request to check:remove entries with N/A mentioned

Hi I have a file with following entries 122 N/A 123 5654656 123423 43534543 4544 45435 435454 N/A i Have to remove entries with N/A so that only 123 5654656 123423 43534543 4544 45435 remain in output file can anybody guide for a code/unix/perl (2 Replies)
Discussion started by: manigrover
2 Replies

9. Shell Programming and Scripting

Request to check:remove entries with blank spaces

Hi I want to remove entries with blank spaces in my input file: 123 234 456 678 56789 345346456 589 3454 345456 3454566............................ (2 Replies)
Discussion started by: manigrover
2 Replies

10. Shell Programming and Scripting

Request to check:remove entries more than once

Hi I have a file like this 1234 2345 567889 567889 2345 234899420 83743 2345 67890 67890 ................ so on I want to delete entries which are more than once like 2345, 567889 and 67890 so that these appear once (4 Replies)
Discussion started by: manigrover
4 Replies
Login or Register to Ask a Question