Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Request to check:remove entries with duplicate numbers in first row

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Request to check:remove entries with duplicate numbers in first row
# 1  
Old 07-31-2012
Request to check:remove entries with duplicate numbers in first row
Hi

I have a file

Code:
1  xyz  456
1  xyz  456
1  xyz  456
2 abc  8459
3  gfd   657
4   ghf   658
4    ghf   658

I want the output

Code:
1  xyz  456
2 abc  8459
3  gfd   657
4   ghf   658

so if first column there is same number duplicated once again , romve the whole next duplicated row.

Please let meknow scripting
Last edited by Scott; 07-31-2012 at 08:40 AM.. Reason: CODE tags, not QUOTE tags
# 2  
Old 07-31-2012
Code:
perl -lne '$x{$_}++; END{print for(sort keys %x)}' file

# 3  
Old 07-31-2012
Request to check
Thanks for reply.

But its not providing expected output

as I think its checking by xyz also may be

I want it shuld check only 1

if one is more than once remove the whole row of second 1

so for example
Code:
1  xyz  456
1  xyz  456
1  xyz  456
2 abc  8459
3  gfd   657
4   ghf   658
4    ghf   658
5 xyz 456
6 gfd  657

the output shuld be

if xyz is more than once but with number 5 it shuld leave it as it is

Code:
1  xyz  456
 2 abc  8459
 3  gfd   657
 4   ghf   658
5 xyz 456
6 gfd  657

Last edited by manigrover; 07-31-2012 at 09:11 AM.. Reason: ...
# 4  
Old 07-31-2012
Try this:
Code:
awk '{s=$0;$1=$1} !a[$0]++{print s}' file

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Check/print missing number in a consecutive range and remove duplicate numbers

Hi, In an ideal scenario, I will have a listing of db transaction log that gets copied to a DR site and if I have them all, they will be numbered consecutively like below. 1_79811_01234567.arc 1_79812_01234567.arc 1_79813_01234567.arc 1_79814_01234567.arc 1_79815_01234567.arc... (3 Replies)
Discussion started by: newbie_01
3 Replies

2. Shell Programming and Scripting

REMOVE DUPLICATE IN a ROW AFTER CHECKING THE FIRST SIMILAR NAME

Hi all I have a big file like this in rows and columns from 2 column onwards the next column is desciption of previous column means 3rd columns is description of 2 columns and 5 column is description of 4 column. All cloumns are separated by comma ... (1 Reply)
Discussion started by: manigrover
1 Replies

3. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

4. Shell Programming and Scripting

Request to check remove duplicates but write before it

Hi alll I have a file with following kind input I want in output duplicates should not be there but there should be numbering mentioned before that like (4 Replies)
Discussion started by: manigrover
4 Replies

5. Shell Programming and Scripting

Request to check:Remove duplicates

Hi all I have a file with following kind of data I want to remove duplicates according to first column so that output contains Kindly let me scripting regading this. (4 Replies)
Discussion started by: manigrover
4 Replies

6. Shell Programming and Scripting

Request to check:remove entries more than once in different column

Hi I have a file 12m 345693460 12 1234 12 1234 34 345 34 345 And I want output fiel as 12m 345693460 12 1234 34 345 hw can it be done Thanks (1 Reply)
Discussion started by: manigrover
1 Replies

7. Shell Programming and Scripting

Request to check:remove entries with N/A mentioned

Hi I have a file with following entries 122 N/A 123 5654656 123423 43534543 4544 45435 435454 N/A i Have to remove entries with N/A so that only 123 5654656 123423 43534543 4544 45435 remain in output file can anybody guide for a code/unix/perl (2 Replies)
Discussion started by: manigrover
2 Replies

8. Shell Programming and Scripting

Request to check:remove entries with blank spaces

Hi I want to remove entries with blank spaces in my input file: 123 234 456 678 56789 345346456 589 3454 345456 3454566............................ (2 Replies)
Discussion started by: manigrover
2 Replies

9. Shell Programming and Scripting

Request to check:remove entries more than once

Hi I have a file like this 1234 2345 567889 567889 2345 234899420 83743 2345 67890 67890 ................ so on I want to delete entries which are more than once like 2345, 567889 and 67890 so that these appear once (4 Replies)
Discussion started by: manigrover
4 Replies

10. Shell Programming and Scripting

Request to check:remove entries with N/A entries

Hi I have a file with numerous entries some entries are 1 mani 2 kavya 3 N/A 4 Praveeen 5 N/A and so on How to remove entries with N/A so the result will be 1 mani 2 kavya 3 Praveeen (6 Replies)
Discussion started by: manigrover
6 Replies
Login or Register to Ask a Question