Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Filtering data using uniq and sed
Top Forums Shell Programming and Scripting Filtering data using uniq and sed Post 302865219 by narachaid on Friday 18th of October 2013 01:41:43 AM
Old 10-18-2013
Filtering data using uniq and sed
Hello,

Does anyone know an easy way to filter this type of file? I want to get everything that has score (column 2) 100.00 and get rid of duplicates (for example gi|332198263|gb|EGK18963.1| below), so I guess uniq can be used for this?

Code:
gi|3379182634|gb|EGK18561.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|332936633|ref|EGK16471.1|	97.00
gi|3329991602|ref|EGK11733.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|332302583|gb|EGK13714.1|	98.00

I want to choose these ones.
Code:
gi|3379182634|gb|EGK18561.1|	100.00
gi|332198263|gb|EGK18963.1|	100.00
gi|3329991602|ref|EGK11733.1|	100.00

In the end, I want the output to look like this.. Is it possible to use sed for this?
OUTPUT:
Code:
EGK18561.1
EGK18963.1
EGK11733.1

Can anyone please help? Thanks so much in advance!
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering out data ...

I have following command which tells me File size in GBs which are greater than 0.01GBs recursively in a dir structure. ls -l -R | awk '{ if ($5/1073741824 >= 0.01) print $9, $5/1073741824 }' But there are some files whom I dont have enough permissions, after executing this script gives me... (1 Reply)
Discussion started by: videsh77
1 Replies

2. Shell Programming and Scripting

Filtering Data

Hi All, I have the below input and expected ouput. I need a code which can scan through this input file and if the number in column1 is more than 1 , it will print out the whole line, else it will output "No Re-occurrence". Can anybody help ? Input: 1 vvvvv 20 7 7 23 0 64 6 zzzzzz 11 5... (7 Replies)
Discussion started by: Raynon
7 Replies

3. UNIX for Dummies Questions & Answers

Filtering Data

file1 contain: (this just a small sample of data it may have thousand of lines) 1 aaa 1/01/1975 delhi 2 bbb 2/03/1977 mumbai 3 ccc 1/01/1975 mumbai 4 ddd 2/03/1977 chennai 5 aaa 1/01/1975 kolkatta 6 bbb 2/03/1977 bangalore program: nawk '{ idx= $2 SUBSEP $3 arr = (idx in arr) ?... (2 Replies)
Discussion started by: bobo
2 Replies

4. Shell Programming and Scripting

How to replicate data using Uniq or awk

Hi, I have this scenario; where there are two classes:- apple and orange. 1,2,3,4,5,6,apple 1,1,0,4,2,3,apple 1,3,3,3,3,4,apple 1,1,1,1,1,1,orange 1,2,3,1,1,1,orange Basically for apple, i have 3 entries in the file, and for orange, I have 2 entries. Im trying to edit the file and find... (5 Replies)
Discussion started by: ahjiefreak
5 Replies

5. Shell Programming and Scripting

help need in filtering data

Hello Gurus, Please help me out of the problem. I ve a input file as below input clock; input a; //reset all input b; //input comment output c; output d; output e; input f; //output comment I need the output as follows: \\Inputs (1 Reply)
Discussion started by: user_prady
1 Replies

6. Shell Programming and Scripting

Help needed with Sort and uniq data

Hi All, After Sorting directories and files i have got following output as below, now i only want the strings common in them, so the actual output should be as below in the bottom. How do i do that? Thanks -adsi File to be modified:- Common Components for ----> AA... (4 Replies)
Discussion started by: asirohi
4 Replies

7. Shell Programming and Scripting

Modify log files to get uniq data

Hello, I have a log file that has following output as below. LAP.sun5 CC LAP.sun5 CQ perl.sun5 CC perl.sun5 CQ TSLogger.sun5 CC TSLogger.sun5 CQ TSLogger.sun5 KR WAS.sun5 CC WAS.sun5 MT WAS.sun5 CQ I want to output to be in the way below, i tried using awk but could not do it. ... (12 Replies)
Discussion started by: asirohi
12 Replies

8. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

9. Shell Programming and Scripting

Filtering out the data with dates

Hi, I have some data like seen below. format : apple(hhmm mm/dd).fruit apple(2345 03/25).fruit apple(2345 05/06).fruit orange(0443 05/02).fruit orange(0345 05/05).fruit orange(2134 05/04).fruit grape(0930 04/24).fruit grape(2330 03/30).fruit I need to get the data which are... (1 Reply)
Discussion started by: jayadanabalan
1 Replies

10. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies

LEARN ABOUT DEBIAN

tm::coverage

TM::Coverage(3pm)					User Contributed Perl Documentation					 TM::Coverage(3pm)

NAME

       TM::Coverage - Topic Maps, Code Coverage

DESCRIPTION

       This auxiliary package keeps track of the code coverage. Probably quite irrelevant for a user.

       Automatically generated for TM (1.44).

	  ---------------------------- ------ ------ ------ ------ ------ ------ ------
	  File				 stmt	bran   cond    sub    pod   time  total
	  ---------------------------- ------ ------ ------ ------ ------ ------ ------
	  blib/lib/TM.pm		 96.4	84.5   58.0   93.7   90.6   27.5   89.6
	  blib/lib/TM/Analysis.pm	100.0	90.9	n/a  100.0  100.0    0.0   97.5
	  blib/lib/TM/AsTMa/Fact.pm	100.0	 n/a	n/a  100.0    n/a    0.0  100.0
	  blib/lib/TM/AsTMa/Fact2.pm	100.0	 n/a	n/a  100.0    n/a    0.0  100.0
	  blib/lib/TM/Bulk.pm		 76.8	71.4   62.5   85.7  100.0    0.0   75.3
	  blib/lib/TM/CTM/CParser.pm	 65.6	34.3   31.0   84.6    0.0    1.6   52.5
	  blib/lib/TM/CTM/Parser.pm	 90.9	50.0	n/a  100.0    0.0    0.0   79.4
	  blib/lib/TM/DM.pm		100.0	65.0   66.7  100.0    n/a    0.0   96.5
	  blib/lib/TM/Graph.pm		 99.1	88.5	n/a  100.0  100.0    0.0   97.4
	  blib/lib/TM/Index.pm		100.0	80.0   60.0  100.0   83.3    0.3   87.8
	  .../Index/Characteristics.pm	100.0  100.0	n/a  100.0    0.0    0.0   97.0
	  blib/lib/TM/Index/Match.pm	 98.5	87.5	n/a  100.0  100.0    0.5   97.6
	  blib/lib/TM/LTM/CParser.pm	 84.9	45.3   42.7   98.2    0.0    0.2   67.8
	  blib/lib/TM/LTM/Parser.pm	 95.7	62.5   50.0  100.0    0.0    0.0   87.1
	  blib/lib/TM/Literal.pm	 45.8	 0.0   11.8   30.0    0.0    0.4   24.2
	  blib/lib/TM/MapSphere.pm	 96.5	84.6   68.8  100.0  100.0    0.0   91.7
	  .../TM/Materialized/AsTMa.pm	100.0	 n/a	n/a  100.0    0.0    0.0   95.5
	  ...ib/TM/Materialized/LTM.pm	100.0	 n/a	n/a  100.0    0.0    0.0   95.5
	  .../TM/Materialized/MLDBM.pm	100.0	83.3	n/a  100.0    0.0    0.0   92.6
	  ...b/TM/Materialized/Null.pm	100.0	 n/a	n/a  100.0    n/a    0.0  100.0
	  ...TM/Materialized/Stream.pm	100.0  100.0  100.0  100.0    0.0    0.0   96.0
	  ...ib/TM/Materialized/XTM.pm	100.0	 n/a	n/a  100.0    0.0    0.0   96.7
	  blib/lib/TM/PSI.pm		100.0	 n/a	n/a  100.0    n/a    0.0  100.0
	  blib/lib/TM/QL.pm		 91.4	61.5   66.7  100.0   50.0    0.1   83.9
	  blib/lib/TM/QL/CParser.pm	 76.2	42.4   41.9   90.3    0.0    9.1   62.0
	  blib/lib/TM/QL/PE.pm		 83.8	77.3   80.0   91.7    0.0   52.5   80.5
	  blib/lib/TM/QL/TS.pm		 65.2	35.4   27.8   70.8   53.3    2.4   56.9
	  blib/lib/TM/ResourceAble.pm	 88.1	87.5	n/a  100.0  100.0    0.0   90.0
	  .../TM/ResourceAble/MLDBM.pm	100.0	75.0	n/a  100.0    0.0    0.0   94.2
	  blib/lib/TM/Serializable.pm	 94.9	65.6  100.0  100.0  100.0    0.0   84.1
	  .../TM/Serializable/AsTMa.pm	 99.1	87.9   66.7  100.0  100.0    0.0   94.2
	  ...TM/Serializable/Dumper.pm	100.0	 n/a	n/a  100.0    0.0    0.0   93.5
	  ...ib/TM/Serializable/LTM.pm	 94.1	 n/a	n/a   83.3  100.0    0.0   92.0
	  ...ib/TM/Serializable/XTM.pm	 96.3	85.2   72.9  100.0  100.0    0.2   91.5
	  .../lib/TM/Synchronizable.pm	100.0	75.0  100.0  100.0  100.0    0.0   97.2
	  ...M/Synchronizable/MLDBM.pm	100.0	50.0	n/a  100.0    0.0    0.0   89.3
	  ...nchronizable/MapSphere.pm	100.0	63.6   64.3  100.0  100.0    0.0   88.6
	  ...TM/Synchronizable/Null.pm	100.0	 n/a	n/a  100.0    0.0    0.0   87.5
	  blib/lib/TM/Tau.pm		100.0	91.7   80.0  100.0    0.0    0.0   95.6
	  blib/lib/TM/Tau/Federate.pm	 63.2	39.3	n/a   62.5   27.3    0.0   54.5
	  blib/lib/TM/Tau/Filter.pm	100.0	91.7   66.7  100.0   50.0    0.0   90.0
	  .../TM/Tau/Filter/Analyze.pm	100.0	 n/a	n/a  100.0  100.0    0.0  100.0
	  blib/lib/TM/Tree.pm		100.0	57.1   44.4  100.0   66.7    0.0   88.0
	  blib/lib/TM/Utils.pm		100.0	50.0	n/a  100.0    0.0    0.0   89.3
	  ...ib/TM/Utils/TreeWalker.pm	100.0  100.0	n/a  100.0    0.0    0.0   96.0
	  ...ib/TM/Workbench/Plugin.pm	100.0	 n/a	n/a  100.0    0.0    0.0   75.0
	  ...M/Workbench/Plugin/Tau.pm	 87.5	50.0	n/a   80.0    0.0    0.0   73.1
	  yapp/astma-fact.yp		 96.9	91.5   73.3   97.8    0.0    4.8   92.7
	  yapp/astma2-fact.yp		 97.4	91.6   58.3  100.0    0.0    0.2   91.6
	  Total 			 76.8	45.2   40.2   91.2   57.0  100.0   63.8
	  ---------------------------- ------ ------ ------ ------ ------ ------ ------

SEE ALSO

       TM

COPYRIGHT AND LICENSE

       Copyright 200[8] by Robert Barta, <drrho@cpan.org>

       This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.

perl v5.10.1							    2008-10-05							 TM::Coverage(3pm)
All times are GMT -4. The time now is 02:24 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy