Sponsored Content
Top Forums Shell Programming and Scripting remove lines based on score criteria Post 302263241 by smriti_shridhar on Monday 1st of December 2008 06:08:41 AM
Old 12-01-2008
Power remove lines based on score criteria

Hi guys,

Please guide for Solution.

PART-I

INPUT FILE (has 2 columns ID and score)
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
BP132435_4 29.5
TC13860_2 10.1

OUTPUT FILE (It shudn't contain the line ' BP132435_4 29.5 ' as BP132435 is repeated and it has lower score. If an ID is repeated more than twice, one with highest score should remain)
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
TC13860_2 10.1

PART-II

====FILE1======
TC5584_1 93.9
DV161411_2 79.5
BP132435_5 46.8
EB682112_1 34.7
TC13860_2 10.1

=====FILE2======
EB681299_3 129 269
EB425502_1 71 182
TC5584_1 66 188
BP132435_5 37 106
EB682112_1 22 150
BP132435_4 117 175
TC13860_2 16 93
DV161411_2 36 239

===OUTPUT_FILE===== (It contains column1 from FILE1 and its corresponding row from FILE2)
TC5584_1 66 188
DV161411_2 36 239
BP132435_5 37 106
EB682112_1 22 150
TC13860_2 16 93

your help is highly appreciated.

Thanks in advance. Smilie

Last edited by smriti_shridhar; 12-01-2008 at 07:22 AM.. Reason: formating
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Append specific lines to a previous line based on sequential search criteria

I'll try explain this as best I can. Let me know if it is not clear. I have large text files that contain data as such: 143593502 09-08-20 09:02:13 xxxxxxxxxxx xxxxxxxxxxx 09-08-20 09:02:11 N line 1 test line 2 test line 3 test 143593503 09-08-20 09:02:13... (3 Replies)
Discussion started by: jesse
3 Replies

2. Shell Programming and Scripting

Delete new lines based on search criteria

Hi all! A bit of background: I am trying to create a script that formats SQL statements. I have gotten so far as to add new lines based on certain match criteria like commas, keywords etc. In the process, I end up adding newlines where I don't want. For example: substr(colName, 1, 10)... (3 Replies)
Discussion started by: jayarkay
3 Replies

3. Shell Programming and Scripting

Remove duplicate lines (the first matching line by field criteria)

Hello to all, I have this file 2002 1 23 0 0 2435.60 131.70 5.60 20.99 0.89 0.00 285.80 2303.90 2002 1 23 15 0 2436.60 132.90 6.45 21.19 1.03 0.00 285.80 2303.70 2002 1 23 ... (6 Replies)
Discussion started by: joggdial3000
6 Replies

4. UNIX for Dummies Questions & Answers

remove duplicates based on a field and criteria

Hi, I have a file with fields like below: A;XYZ;102345;222 B;XYZ;123243;333 C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 desired output: C;ABC;234234;444 D;MNO;103345;222 E;DEF;124243;333 ie, if the 4rth field is a duplicate.. i need only those records where... (5 Replies)
Discussion started by: wanderingmind16
5 Replies

5. Shell Programming and Scripting

Merging Lines based on criteria

Hello, Need help with following scenario. A file contains following text: {beginning of file} New: This is a new record and it is not on same line. Since I have lost touch with script take this challenge and bring all this in one line. New: Hello losttouch. You seem to be struggling... (4 Replies)
Discussion started by: losttouch
4 Replies

6. Shell Programming and Scripting

Select lines from a file based on a criteria

Hi I need to select lines from a txt file, I have got a line starting with ZMIO:MSISDN= and after a few line I have another line starting with 'MOBILE STATION ISDN NUMBER' and another one starting with 'VLR-ADDRESS' I need to copy these three lines as three different columns in a separate... (3 Replies)
Discussion started by: Tlcm sam
3 Replies

7. Shell Programming and Scripting

Need To Delete Lines Based On Search Criteria

Hi All, I have following input file. I wish to retain those lines which match multiple search criteria. The search criteria is stored in a variable seperated from each other by comma(,). SEARCH_CRITERIA = "REJECT, DUPLICATE" Input File: ERROR,MYFILE_20130214_11387,9,37.75... (3 Replies)
Discussion started by: angshuman
3 Replies

8. Shell Programming and Scripting

Split a file in more files based on score content

Dear All, I have the following file tabulated: ID distanceTSS score 8434 571269 10 10122 393912 9 7652 6 10 4863 1451 9 8419 39 2 9363 564 21 9333 7714 22 9638 8334 9 1638 1231 11 10701 918 1000 6587 32056 111 What I would like to do is the following, create 100 new files based... (5 Replies)
Discussion started by: paolo.kunder
5 Replies

9. Shell Programming and Scripting

Two files, remove lines from second based on lines in first

I have two files, a keepout.txt and a database.csv. They're unsorted, but could be sorted. keepout: user1 buser3 anuser19 notheruser27 database: user1,2343,"information about",field,blah,34 user2,4231,"mo info",etc,stuff,43 notheruser27,4344,"hiya",thing,more thing,423... (4 Replies)
Discussion started by: esoffron
4 Replies

10. UNIX for Beginners Questions & Answers

Remove lines from File.A based on criteria in File.B

Hello, I have two files of the following form. I would like to remove from File.A where the first three colum matches values in File.B to give the output in File.C File.A 121 54321 PQR CAT 122 765431 ABC DOG 124 98765 ZXY TIGER 125 86432 GEF LION File.B 122 765431 ABC 125 86432 GEF... (4 Replies)
Discussion started by: Gussifinknottle
4 Replies
TEST(1) 							   User Commands							   TEST(1)

NAME
test - check file types and compare values SYNOPSIS
test EXPRESSION test [ EXPRESSION ] [ ] [ OPTION DESCRIPTION
Exit with the status determined by EXPRESSION. --help display this help and exit --version output version information and exit An omitted EXPRESSION defaults to false. Otherwise, EXPRESSION is true or false and sets exit status. It is one of: ( EXPRESSION ) EXPRESSION is true ! EXPRESSION EXPRESSION is false EXPRESSION1 -a EXPRESSION2 both EXPRESSION1 and EXPRESSION2 are true EXPRESSION1 -o EXPRESSION2 either EXPRESSION1 or EXPRESSION2 is true -n STRING the length of STRING is nonzero STRING equivalent to -n STRING -z STRING the length of STRING is zero STRING1 = STRING2 the strings are equal STRING1 != STRING2 the strings are not equal INTEGER1 -eq INTEGER2 INTEGER1 is equal to INTEGER2 INTEGER1 -ge INTEGER2 INTEGER1 is greater than or equal to INTEGER2 INTEGER1 -gt INTEGER2 INTEGER1 is greater than INTEGER2 INTEGER1 -le INTEGER2 INTEGER1 is less than or equal to INTEGER2 INTEGER1 -lt INTEGER2 INTEGER1 is less than INTEGER2 INTEGER1 -ne INTEGER2 INTEGER1 is not equal to INTEGER2 FILE1 -ef FILE2 FILE1 and FILE2 have the same device and inode numbers FILE1 -nt FILE2 FILE1 is newer (modification date) than FILE2 FILE1 -ot FILE2 FILE1 is older than FILE2 -b FILE FILE exists and is block special -c FILE FILE exists and is character special -d FILE FILE exists and is a directory -e FILE FILE exists -f FILE FILE exists and is a regular file -g FILE FILE exists and is set-group-ID -G FILE FILE exists and is owned by the effective group ID -h FILE FILE exists and is a symbolic link (same as -L) -k FILE FILE exists and has its sticky bit set -L FILE FILE exists and is a symbolic link (same as -h) -O FILE FILE exists and is owned by the effective user ID -p FILE FILE exists and is a named pipe -r FILE FILE exists and read permission is granted -s FILE FILE exists and has a size greater than zero -S FILE FILE exists and is a socket -t FD file descriptor FD is opened on a terminal -u FILE FILE exists and its set-user-ID bit is set -w FILE FILE exists and write permission is granted -x FILE FILE exists and execute (or search) permission is granted Except for -h and -L, all FILE-related tests dereference symbolic links. Beware that parentheses need to be escaped (e.g., by backslashes) for shells. INTEGER may also be -l STRING, which evaluates to the length of STRING. NOTE: [ honors the --help and --version options, but test does not. test treats each of those as it treats any other nonempty STRING. NOTE: your shell may have its own version of test and/or [, which usually supersedes the version described here. Please refer to your shell's documentation for details about the options it supports. AUTHOR
Written by Kevin Braunsdorf and Matthew Bradburn. REPORTING BUGS
Report test bugs to bug-coreutils@gnu.org GNU coreutils home page: <http://www.gnu.org/software/coreutils/> General help using GNU software: <http://www.gnu.org/gethelp/> COPYRIGHT
Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
The full documentation for test is maintained as a Texinfo manual. If the info and test programs are properly installed at your site, the command info coreutils 'test invocation' should give you access to the complete manual. GNU coreutils 7.1 July 2010 TEST(1)
All times are GMT -4. The time now is 11:43 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy