Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Filtering data using uniq and sed
Top Forums Shell Programming and Scripting Filtering data using uniq and sed Post 302865219 by narachaid on Friday 18th of October 2013 01:41:43 AM
Old 10-18-2013
Filtering data using uniq and sed
Hello,

Does anyone know an easy way to filter this type of file? I want to get everything that has score (column 2) 100.00 and get rid of duplicates (for example gi|332198263|gb|EGK18963.1| below), so I guess uniq can be used for this?

Code:
gi|3379182634|gb|EGK18561.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|332936633|ref|EGK16471.1|	97.00
gi|3329991602|ref|EGK11733.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|332302583|gb|EGK13714.1|	98.00

I want to choose these ones.
Code:
gi|3379182634|gb|EGK18561.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|3329991602|ref|EGK11733.1|	100.00

In the end, I want the output to look like this.. Is it possible to use sed for this?
OUTPUT:
Code:
EGK18561.1
EGK18963.1
EGK11733.1

Can anyone please help? Thanks so much in advance!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering out data ...

I have following command which tells me File size in GBs which are greater than 0.01GBs recursively in a dir structure. ls -l -R | awk '{ if ($5/1073741824 >= 0.01) print $9, $5/1073741824 }' But there are some files whom I dont have enough permissions, after executing this script gives me... (1 Reply)
Discussion started by: videsh77
1 Replies

2. Shell Programming and Scripting

Filtering Data

Hi All, I have the below input and expected ouput. I need a code which can scan through this input file and if the number in column1 is more than 1 , it will print out the whole line, else it will output "No Re-occurrence". Can anybody help ? Input: 1 vvvvv 20 7 7 23 0 64 6 zzzzzz 11 5... (7 Replies)
Discussion started by: Raynon
7 Replies

3. UNIX for Dummies Questions & Answers

Filtering Data

file1 contain: (this just a small sample of data it may have thousand of lines) 1 aaa 1/01/1975 delhi 2 bbb 2/03/1977 mumbai 3 ccc 1/01/1975 mumbai 4 ddd 2/03/1977 chennai 5 aaa 1/01/1975 kolkatta 6 bbb 2/03/1977 bangalore program: nawk '{ idx= $2 SUBSEP $3 arr = (idx in arr) ?... (2 Replies)
Discussion started by: bobo
2 Replies

4. Shell Programming and Scripting

How to replicate data using Uniq or awk

Hi, I have this scenario; where there are two classes:- apple and orange. 1,2,3,4,5,6,apple 1,1,0,4,2,3,apple 1,3,3,3,3,4,apple 1,1,1,1,1,1,orange 1,2,3,1,1,1,orange Basically for apple, i have 3 entries in the file, and for orange, I have 2 entries. Im trying to edit the file and find... (5 Replies)
Discussion started by: ahjiefreak
5 Replies

5. Shell Programming and Scripting

help need in filtering data

Hello Gurus, Please help me out of the problem. I ve a input file as below input clock; input a; //reset all input b; //input comment output c; output d; output e; input f; //output comment I need the output as follows: \\Inputs (1 Reply)
Discussion started by: user_prady
1 Replies

6. Shell Programming and Scripting

Help needed with Sort and uniq data

Hi All, After Sorting directories and files i have got following output as below, now i only want the strings common in them, so the actual output should be as below in the bottom. How do i do that? Thanks -adsi File to be modified:- Common Components for ----> AA... (4 Replies)
Discussion started by: asirohi
4 Replies

7. Shell Programming and Scripting

Modify log files to get uniq data

Hello, I have a log file that has following output as below. LAP.sun5 CC LAP.sun5 CQ perl.sun5 CC perl.sun5 CQ TSLogger.sun5 CC TSLogger.sun5 CQ TSLogger.sun5 KR WAS.sun5 CC WAS.sun5 MT WAS.sun5 CQ I want to output to be in the way below, i tried using awk but could not do it. ... (12 Replies)
Discussion started by: asirohi
12 Replies

8. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

9. Shell Programming and Scripting

Filtering out the data with dates

Hi, I have some data like seen below. format : apple(hhmm mm/dd).fruit apple(2345 03/25).fruit apple(2345 05/06).fruit orange(0443 05/02).fruit orange(0345 05/05).fruit orange(2134 05/04).fruit grape(0930 04/24).fruit grape(2330 03/30).fruit I need to get the data which are... (1 Reply)
Discussion started by: jayadanabalan
1 Replies

10. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

LEARN ABOUT MOJAVE

perlio::via::symlink5.18

symlink(3)						User Contributed Perl Documentation						symlink(3)

NAME

       PerlIO::via::symlink - PerlIO layers for create symlinks

SYNOPSIS

	open $fh, '>:via(symlink)', $fname;
	print $fh "link foobar";
	close $fh;

DESCRIPTION

       The PerlIO layer "symlink" allows you to create a symbolic link by writing to the file handle.

       You need to write C"link $name" to the file handle. If the format does not match, "close" will fail with EINVAL.

TEST COVERAGE

	----------------------------------- ------ ------ ------ ------ ------ ------
	File				      stmt branch   cond    sub   time	total
	----------------------------------- ------ ------ ------ ------ ------ ------
	blib/lib/PerlIO/via/symlink.pm	     100.0  100.0    n/a  100.0  100.0	100.0
	Total				     100.0  100.0    n/a  100.0  100.0	100.0
	----------------------------------- ------ ------ ------ ------ ------ ------

AUTHORS

       Chia-liang Kao <clkao@clkao.org>

COPYRIGHT

       Copyright 2004-2005 by Chia-liang Kao <clkao@clkao.org>.

       This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

       See <http://www.perl.com/perl/misc/Artistic.html>

perl v5.18.2							    2005-03-01								symlink(3)
All times are GMT -4. The time now is 06:46 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy