Sponsored Content
Top Forums Shell Programming and Scripting Filtering data using uniq and sed Post 302865225 by Akshay Hegde on Friday 18th of October 2013 01:58:11 AM
Old 10-18-2013
You may try awk solution its easy, Others might have sed solution

Code:
$ awk -F'[|]' '$5==100 && !A[$4]++ {print $4}' file

OR
Code:
$ grep -E '(100)'  file | cut -d '|' -f4 | sort | uniq

Resulting
Code:
EGK18561.1
EGK18963.1
EGK11733.1


Last edited by Akshay Hegde; 10-18-2013 at 03:05 AM..
This User Gave Thanks to Akshay Hegde For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Filtering out data ...

I have following command which tells me File size in GBs which are greater than 0.01GBs recursively in a dir structure. ls -l -R | awk '{ if ($5/1073741824 >= 0.01) print $9, $5/1073741824 }' But there are some files whom I dont have enough permissions, after executing this script gives me... (1 Reply)
Discussion started by: videsh77
1 Replies

2. Shell Programming and Scripting

Filtering Data

Hi All, I have the below input and expected ouput. I need a code which can scan through this input file and if the number in column1 is more than 1 , it will print out the whole line, else it will output "No Re-occurrence". Can anybody help ? Input: 1 vvvvv 20 7 7 23 0 64 6 zzzzzz 11 5... (7 Replies)
Discussion started by: Raynon
7 Replies

3. UNIX for Dummies Questions & Answers

Filtering Data

file1 contain: (this just a small sample of data it may have thousand of lines) 1 aaa 1/01/1975 delhi 2 bbb 2/03/1977 mumbai 3 ccc 1/01/1975 mumbai 4 ddd 2/03/1977 chennai 5 aaa 1/01/1975 kolkatta 6 bbb 2/03/1977 bangalore program: nawk '{ idx= $2 SUBSEP $3 arr = (idx in arr) ?... (2 Replies)
Discussion started by: bobo
2 Replies

4. Shell Programming and Scripting

How to replicate data using Uniq or awk

Hi, I have this scenario; where there are two classes:- apple and orange. 1,2,3,4,5,6,apple 1,1,0,4,2,3,apple 1,3,3,3,3,4,apple 1,1,1,1,1,1,orange 1,2,3,1,1,1,orange Basically for apple, i have 3 entries in the file, and for orange, I have 2 entries. Im trying to edit the file and find... (5 Replies)
Discussion started by: ahjiefreak
5 Replies

5. Shell Programming and Scripting

help need in filtering data

Hello Gurus, Please help me out of the problem. I ve a input file as below input clock; input a; //reset all input b; //input comment output c; output d; output e; input f; //output comment I need the output as follows: \\Inputs (1 Reply)
Discussion started by: user_prady
1 Replies

6. Shell Programming and Scripting

Help needed with Sort and uniq data

Hi All, After Sorting directories and files i have got following output as below, now i only want the strings common in them, so the actual output should be as below in the bottom. How do i do that? Thanks -adsi File to be modified:- Common Components for ----> AA... (4 Replies)
Discussion started by: asirohi
4 Replies

7. Shell Programming and Scripting

Modify log files to get uniq data

Hello, I have a log file that has following output as below. LAP.sun5 CC LAP.sun5 CQ perl.sun5 CC perl.sun5 CQ TSLogger.sun5 CC TSLogger.sun5 CQ TSLogger.sun5 KR WAS.sun5 CC WAS.sun5 MT WAS.sun5 CQ I want to output to be in the way below, i tried using awk but could not do it. ... (12 Replies)
Discussion started by: asirohi
12 Replies

8. UNIX for Dummies Questions & Answers

Finding and Extracting uniq data in multiple files

Hi, I have several files that look like this: File1.txt Data1 Data2 Data20 File2.txt Data1 Data5 Data10 File3.txt Data1 Data2 Data17 File4.txt (6 Replies)
Discussion started by: Fahmida
6 Replies

9. Shell Programming and Scripting

Filtering out the data with dates

Hi, I have some data like seen below. format : apple(hhmm mm/dd).fruit apple(2345 03/25).fruit apple(2345 05/06).fruit orange(0443 05/02).fruit orange(0345 05/05).fruit orange(2134 05/04).fruit grape(0930 04/24).fruit grape(2330 03/30).fruit I need to get the data which are... (1 Reply)
Discussion started by: jayadanabalan
1 Replies

10. Shell Programming and Scripting

Combine data from two files base on uniq data

File 1 ID Name Po1 Po2 DD134 DD134_4A_1 NN-1 L_0_1 DD134 DD134_4B_1 NN-2 L_1_1 DD134 DD134_4C_1 NN-3 L_2_1 DD142 DD142_4A_1 NN-1 L_0_1 DD142 DD142_4B_1 NN-2 L_1_1 DD142 DD142_4C_1 NN-3 L_2_1 DD142 DD142_3A_1 NN-41 L_3_1 DD142 DD142_3A_1 NN-42 L_3_2 File 2 ( Combination of... (1 Reply)
Discussion started by: pareshkp
1 Replies
Mail::SpamAssassin::Plugin::Shortcircuit(3pm)		User Contributed Perl Documentation	     Mail::SpamAssassin::Plugin::Shortcircuit(3pm)

NAME
Mail::SpamAssassin::Plugin::Shortcircuit - short-circuit evaluation for certain rules SYNOPSIS
loadplugin Mail::SpamAssassin::Plugin::Shortcircuit report Content analysis details: (_SCORE_ points, _REQD_ required, s/c _SCTYPE_) add_header all Status "_YESNO_, score=_SCORE_ required=_REQD_ tests=_TESTS_ shortcircuit=_SCTYPE_ autolearn=_AUTOLEARN_ version=_VERSION_" DESCRIPTION
This plugin implements simple, test-based shortcircuiting. Shortcircuiting a test will force all other pending rules to be skipped, if that test is hit. In addition, a symbolic rule, "SHORTCIRCUIT", will fire. Recomended usage is to use "priority" to set rules with strong S/O values (ie. 1.0) to be run first, and make instant spam or ham classification based on that. CONFIGURATION SETTINGS
The following configuration settings are used to control shortcircuiting: shortcircuit SYMBOLIC_TEST_NAME {ham|spam|on|off} Shortcircuiting a test will force all other pending rules to be skipped, if that test is hit. Recomended usage is to use "priority" to set rules with strong S/O values (ie. 1.0) to be run first, and make instant spam or ham classification based on that. To override a test that uses shortcircuiting, you can set the classification type to "off". on Shortcircuits the rest of the tests, but does not make a strict classification of spam or ham. Rather, it uses the default score for the rule being shortcircuited. This would allow you, for example, to define a rule such as body TEST /test/ describe TEST test rule that scores barely over spam threshold score TEST 5.5 priority TEST -100 shortcircuit TEST on The result of a message hitting the above rule would be a final score of 5.5, as opposed to 100 (default) if it were classified as spam. off Disables shortcircuiting on said rule. spam Shortcircuit the rule using a set of defaults; override the default score of this rule with the score from "shortcircuit_spam_score", set the "noautolearn" tflag, and set priority to "-100". In other words, equivalent to: shortcircuit TEST on priority TEST -100 score TEST 100 tflags TEST noautolearn ham Shortcircuit the rule using a set of defaults; override the default score of this rule with the score from "shortcircuit_ham_score", set the "noautolearn" and "nice" tflags, and set priority to "-100". In other words, equivalent to: shortcircuit TEST on priority TEST -100 score TEST -100 tflags TEST noautolearn nice shortcircuit_spam_score n.nn (default: 100) When shortcircuit is used on a rule, and the shortcircuit classification type is set to "spam", this value should be applied in place of the default score for that rule. shortcircuit_ham_score n.nn (default: -100) When shortcircuit is used on a rule, and the shortcircuit classification type is set to "ham", this value should be applied in place of the default score for that rule. TAGS
The following tags are added to the set available for use in reports, headers etc.: _SC_ shortcircuit status (classification and rule name) _SCRULE_ rulename that caused the shortcircuit _SCTYPE_ shortcircuit classification ("spam", "ham", "default", "none") SEE ALSO
"http://issues.apache.org/SpamAssassin/show_bug.cgi?id=3109" perl v5.14.2 2011-06-06 Mail::SpamAssassin::Plugin::Shortcircuit(3pm)
All times are GMT -4. The time now is 06:34 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy