Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Request to check:remove entries more than once

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Request to check:remove entries more than once
# 1  
Old 05-10-2012
Request to check:remove entries more than once
Hi

I have a file like this

Code:
1234
2345
567889
567889
2345
234899420
83743
2345
67890
67890

................
so on

I want to delete entries which are more than once like 2345, 567889 and 67890 so that these appear once

and output shuld contain
Code:
1234
2345
567889
234899420
83743
67890

So no repeted entry please let me know Unix code/perl script regarding this
# 2  
Old 05-10-2012
Code:
 
sort -u filename.txt

This User Gave Thanks to itkamaraj For This Post:
manigrover
# 3  
Old 05-10-2012
In the same order as it appears in the inputfile:

Code:
perl -lne 'chomp; if(!defined $x{$_}){$x{$_}=1;print}' inputfile

This User Gave Thanks to balajesuri For This Post:
manigrover
# 4  
Old 05-10-2012
Code:
awk '!a[$0]++' input.txt

This User Gave Thanks to itkamaraj For This Post:
manigrover
# 5  
Old 05-10-2012
one more in awk.
Code:
$ awk '!($0 in a) {a[$0];print}' infile

This User Gave Thanks to jayan_jay For This Post:
manigrover
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Request to check:Fetch certain entries

Hi all. Kindly check it it's urgent!! I have one big file from which which I have to fetch certain data I have attached a small part of this file. from the attached file, I have to fetch and arrange data in 3 columns 1 Generic name 2. Brand names 3. Drug... (10 Replies)
Discussion started by: manigrover
10 Replies

2. Shell Programming and Scripting

Request to check:remove duplicates only in first column

Hi all, I have an input file like this Now I have to remove duplicates only in first column and nothing has to be changed in second and third column. so that output would be Please let me know scripting regarding this (20 Replies)
Discussion started by: manigrover
20 Replies

3. Shell Programming and Scripting

Request to check:remove entries with duplicate numbers in first row

Hi I have a file 1 xyz 456 1 xyz 456 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 4 ghf 658 I want the output 1 xyz 456 2 abc 8459 3 gfd 657 4 ghf 658 (3 Replies)
Discussion started by: manigrover
3 Replies

4. Shell Programming and Scripting

Request to check:remove duplicates and write sytematically

Hi all I have a file with following input It contains 5 columns gene name drug drug ID disease approved Now the same gene is repeated many times with different data in column2,3 ,4,5 I want to arrange dat in such a way that there shuld be one entry in the column(no... (2 Replies)
Discussion started by: manigrover
2 Replies

5. Shell Programming and Scripting

Request to check remove duplicates but write before it

Hi alll I have a file with following kind input I want in output duplicates should not be there but there should be numbering mentioned before that like (4 Replies)
Discussion started by: manigrover
4 Replies

6. Shell Programming and Scripting

Request to check:Remove duplicates

Hi all I have a file with following kind of data I want to remove duplicates according to first column so that output contains Kindly let me scripting regading this. (4 Replies)
Discussion started by: manigrover
4 Replies

7. Shell Programming and Scripting

Request to check:remove entries more than once in different column

Hi I have a file 12m 345693460 12 1234 12 1234 34 345 34 345 And I want output fiel as 12m 345693460 12 1234 34 345 hw can it be done Thanks (1 Reply)
Discussion started by: manigrover
1 Replies

8. Shell Programming and Scripting

Request to check:remove entries with N/A mentioned

Hi I have a file with following entries 122 N/A 123 5654656 123423 43534543 4544 45435 435454 N/A i Have to remove entries with N/A so that only 123 5654656 123423 43534543 4544 45435 remain in output file can anybody guide for a code/unix/perl (2 Replies)
Discussion started by: manigrover
2 Replies

9. Shell Programming and Scripting

Request to check:remove entries with blank spaces

Hi I want to remove entries with blank spaces in my input file: 123 234 456 678 56789 345346456 589 3454 345456 3454566............................ (2 Replies)
Discussion started by: manigrover
2 Replies

10. Shell Programming and Scripting

Request to check:remove entries with N/A entries

Hi I have a file with numerous entries some entries are 1 mani 2 kavya 3 N/A 4 Praveeen 5 N/A and so on How to remove entries with N/A so the result will be 1 mani 2 kavya 3 Praveeen (6 Replies)
Discussion started by: manigrover
6 Replies
Login or Register to Ask a Question