Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Filtering a file for unique entries

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Filtering a file for unique entries
# 1  
Old 09-21-2011
Filtering a file for unique entries
Hi All,

I am trying to filter out a txt file containing 2 columns. Which has got website name and response time as below.
Website_name response_time
yahoo 22
google 21
yahoo 34
rediff 45
google 43
rediff 31

I want filter the above file with unique website names which has got highest response time. can you please let me know ksh script for doing this.
# 2  
Old 09-21-2011
Code:
awk '$2>a[$1]{a[$1]=$2}END{for (i in a) print i,a[i]}' file

# 3  
Old 09-21-2011
Code:
SCRIPTS>awk '{ x=a[$1];$2>a[$1]?a[$1]=$2:a[$1]=x } END { for(i in a) print i,a[i]}' input_file

# 4  
Old 09-21-2011
Thanks bartus for the quick response.

Also I want to filter those which have got response time greater than a given value. Forgot to add this to the post.
# 5  
Old 09-21-2011
Code:
awk '$2>a[$1]{a[$1]=$2}END{for (i in a) if (a[i]>some_value) print i,a[i]}' file

# 6  
Old 09-21-2011
Thank you bartus Smilie. It worked. Thanks Panyam.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Print unique lines without sort or unique

I would like to print unique lines without sort or unique. Unfortunately the server I am working on does not have sort or unique. I have not been able to contact the administrator of the server to ask him to add it for several weeks. (7 Replies)
Discussion started by: cokedude
7 Replies

2. Shell Programming and Scripting

Filtering first file columns based on second file column

Hi friends, I have one file like below. (.csv type) SNo,data1,data2 1,1,2 2,2,3 3,3,2 and another file like below. Exclude data1 where Exclude should be treated as column name in file2. I want the output shown below. SNo,data2 1,2 2,3 3,2 Where my data1 column got removed from... (2 Replies)
Discussion started by: ks_reddy
2 Replies

3. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

4. Shell Programming and Scripting

unique entries

Hi, I have a file ENST12345 yes ENST11445 no ENST12345 yes ENST24689 no ENST24689 yes ENST34567 no what I want is for each unique identifier if atleast one of them has a value of "yes" then all of it entries need to be imported or else it should be skipped. o/p... (2 Replies)
Discussion started by: Diya123
2 Replies

5. UNIX for Dummies Questions & Answers

Filtering records from 1 file based on some manipulation doen on second file

Hi, I am looking for an awk script which should help me to meet the following requirement: File1 has records in following format INF: FAILEd RECORD AB1234 INF: FAILEd RECORD PQ1145 INF: FAILEd RECORD AB3215 INF: FAILEd RECORD AB6114 ............................ (2 Replies)
Discussion started by: mintu41
2 Replies

6. Shell Programming and Scripting

unique entries needed

Hi All, I have the following command which returns the output like this - wget URL|grep hostname|awk '{print $1,$3}'|/usr/bin/perl -nle 'm{"(.*?)".*?//(.*):(\d+)/} and print "$1 $2 $3"'|while read host port do echo check_env_$port echo host done OUTPUT Appname hostname port Appname2... (1 Reply)
Discussion started by: jacki
1 Replies

7. Homework & Coursework Questions

Filtering Unique Lines

Use and complete the template provided. The entire template must be completed. If you don't, your post may be deleted! 1. The problem statement, all variables and given/known data: The uniq command excludes consecutive duplicate lines. It has a -c option to display a count of the number... (1 Reply)
Discussion started by: billydeanmak
1 Replies

8. UNIX for Dummies Questions & Answers

Filtering a file

I have a list of directories looking something like; /usr/local/1/in /usr/local/1/out /usr/local/1/archive /usr/local/2/in /usr/local/2/out /usr/local/2/archive /usr/local/3/in /usr/local/3/out /usr/local/3/archive Is there a way I can filter the out and archive directories so I... (5 Replies)
Discussion started by: JayC89
5 Replies

9. Shell Programming and Scripting

get part of file with unique & non-unique string

I have an archive file that holds a batch of statements. I would like to be able to extract a certain statement based on the unique customer # (ie. 123456). The end for each statement is noted by "ENDSTM". I can find the line number for the beginning of the statement section with sed. ... (5 Replies)
Discussion started by: andrewsc
5 Replies

10. Shell Programming and Scripting

Filtering multiple entries in a file

Hi, I have a requirement to filter multiple entries in a text file. Entries are a pair of 2 lines .Like line1 line2 ....... ........ Here line1, line2 consist of one pair. line1 and line2 contain different data. There can be multiple entries for the same pair. And there can be 'n' such... (2 Replies)
Discussion started by: suman_jakkula
2 Replies
Login or Register to Ask a Question