12-01-2008
remove lines based on score criteria
Hi guys,
Please guide for Solution.
PART-I
INPUT FILE (has 2 columns ID and score)
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
BP132435_4 29.5
TC13860_2 10.1
OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated and it has lower score. If an ID is repeated more than twice, one with highest score should remain)
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
TC13860_2 10.1
PART-II
====FILE1======
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
TC13860_2 10.1
=====FILE2======
EB681299_3 129 269
EB425502_1 71 182
TC5584_1 66 188
BP132435_5 37 106
EB682112_1 22 150
BP132435_4 117 175
TC13860_2 16 93
DV161411_2 36 239
===OUTPUT_FILE===== (It contains column1 from FILE1 and its corresponding row from FILE2)
TC5584_1 66 188
DV161411_2 36 239
BP132435_5 37 106
EB682112_1 22 150
TC13860_2 16 93
your help is highly appreciated.
Thanks in advance.
Last edited by smriti_shridhar; 12-01-2008 at 07:22 AM..
Reason: formating
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I'll try explain this as best I can. Let me know if it is not clear.
I have large text files that contain data as such:
143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test
line 2 test
line 3 test
143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies
2. Shell Programming and Scripting
Hi all!
A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want.
For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies
3. Shell Programming and Scripting
Hello to all,
I have this file
2002 1 23 0 0 2435.60 131.70 5.60 20.99 0.89 0.00 285.80 2303.90
2002 1 23 15 0 2436.60 132.90 6.45 21.19 1.03 0.00 285.80 2303.70
2002 1 23 ... (6 Replies)
Discussion started by: joggdial3000
6 Replies
4. UNIX for Dummies Questions & Answers
Hi,
I have a file with fields like below:
A;XYZ;102345;222
B;XYZ;123243;333
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
desired output:
C;ABC;234234;444
D;MNO;103345;222
E;DEF;124243;333
ie, if the 4rth field is a duplicate.. i need only those records where... (5 Replies)
Discussion started by: wanderingmind16
5 Replies
5. Shell Programming and Scripting
Hello,
Need help with following scenario.
A file contains following text:
{beginning of file}
New: This is a new record and it is not
on same line. Since I have lost touch with script
take this challenge and bring all this in one line.
New: Hello losttouch. You seem to be struggling... (4 Replies)
Discussion started by: losttouch
4 Replies
6. Shell Programming and Scripting
Hi
I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies
7. Shell Programming and Scripting
Hi All,
I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,).
SEARCH_CRITERIA = "REJECT, DUPLICATE"
Input File:
ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies
8. Shell Programming and Scripting
Dear All,
I have the following file tabulated:
ID distanceTSS score
8434 571269 10
10122 393912 9
7652 6 10
4863 1451 9
8419 39 2
9363 564 21
9333 7714 22
9638 8334 9
1638 1231 11
10701 918 1000
6587 32056 111
What I would like to do is the following, create 100 new files based... (5 Replies)
Discussion started by: paolo.kunder
5 Replies
9. Shell Programming and Scripting
I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted.
keepout:
user1
buser3
anuser19
notheruser27
database:
user1,2343,"information about",field,blah,34
user2,4231,"mo info",etc,stuff,43
notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies
10. UNIX for Beginners Questions & Answers
Hello,
I have two files of the following form. I would like to remove from File.A where the first three colum matches values in File.B to give the output in File.C
File.A
121 54321 PQR CAT
122 765431 ABC DOG
124 98765 ZXY TIGER
125 86432 GEF LION
File.B
122 765431 ABC
125 86432 GEF... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies
LEARN ABOUT REDHAT
mail::spamassassin::persistentaddrlist
Mail::SpamAssassin::PersistentAddrList(3) User Contributed Perl Documentation Mail::SpamAssassin::PersistentAddrList(3)
NAME
Mail::SpamAssassin::PersistentAddrList - persistent address list base class
SYNOPSIS
my $factory = PersistentAddrListSubclass->new();
$spamtest->set_persistent_addr_list_factory ($factory);
... call into SpamAssassin classes...
SpamAssassin will call:
my $addrlist = $factory->new_checker($spamtest);
$entry = $addrlist->get_addr_entry ($addr);
...
DESCRIPTION
All persistent address list implementations, used by the auto-whitelist code to track known-good email addresses, use this as a base class.
METHODS
$factory = PersistentAddrListSubclass->new();
This creates a factory object, which SpamAssassin will call to create a new checker object for the persistent address list.
my $addrlist = $factory->new_checker();
Create a new address-list checker object from the factory. Called by the SpamAssassin classes.
$entry = $addrlist->get_addr_entry ($addr);
Given an email address $addr, return an entry object with the details of that address.
The entry object is a reference to a hash, which must contain at least two keys: "count", which is the count of times that address has
been encountered before; and "totscore", which is the total of all scores for messages associated with that address. From these two
fields, an average score will be calculated, and the score for the current message will be regressed towards that mean message score.
The hash can contain whatever other data your back-end needs to store, under other keys.
The method should never return "undef", or a hash that does not contain a "count" key and a "totscore" key.
$entry = $addrlist->add_score($entry, $score);
This method should add the given score to the whitelist database for the given entry, and then return the new entry.
$entry = $addrlist->remove_entry ($entry);
This method should remove the given entry from the whitelist database.
$entry = $addrlist->finish ();
Clean up, if necessary. Called by SpamAssassin when it has finished checking, or adding to, the auto-whitelist database.
perl v5.8.0 2002-02-19 Mail::SpamAssassin::PersistentAddrList(3)