Sponsored Content
Top Forums UNIX for Dummies Questions & Answers remove duplicate lines based on two columns and judging from a third one Post 302555296 by sk1418 on Wednesday 14th of September 2011 10:30:21 AM
Old 09-14-2011
for negative number in column3:



Code:
kent$  awk '{f=substr($1,1,4)$4;if((f in a && $3<b[f])||(! (f in a))){ a[f] = $0;b[f]=$3; } } END{for (x in a)print a[x]} yourFile

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove lines, Sorted with Time based columns using AWK & SORT

Hi having a file as follows MediaErr.log 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:12:16 84 Server1 Policy1 Schedule1 master1 05/08/2008 02:22:47 84 Server1 Policy1 Schedule1 master1 05/08/2008 03:41:26 84 Server1 Policy1 ... (1 Reply)
Discussion started by: karthikn7974
1 Replies

2. UNIX for Dummies Questions & Answers

Duplicate columns and lines

Hi all, I have a tab-delimited file and want to remove identical lines, i.e. all of line 1,2,4 because the columns are the same as the columns in other lines. Any input is appreciated. abc gi4597 9997 cgcgtgcg $%^&*()()* abc gi4597 9997 cgcgtgcg $%^&*()()* ttt ... (1 Reply)
Discussion started by: dr_sabz
1 Replies

3. Shell Programming and Scripting

Remove duplicate columns in input file

hello, I have an input file which looks like this: 2 C:G 17 -0.14 8.75 33.35 3 G:C 16 -2.28 0.98 28.22 4 C:G 15 0.39 11.06 29.31 5 G:C 14 2.64 5.17 36.07 6 G:C 13 -0.65 2.05 21.94 7 C:G 11 138.96 21.64 14.40 9 C:G 27 -2.40 6.95 27.98 10 C:G 26 2.89 15.60 34.33 11 G:C... (7 Replies)
Discussion started by: linux_usr
7 Replies

4. Shell Programming and Scripting

Remove duplicate lines based on field and sort

I have a csv file that I would like to remove duplicate lines based on field 1 and sort. I don't care about any of the other fields but I still wanna keep there data intact. I was thinking I could do something like this but I have no idea how to print the full line with this. Please show any method... (8 Replies)
Discussion started by: cokedude
8 Replies

5. Shell Programming and Scripting

Remove duplicate based on Group

Hi, How can I remove duplicates from a file based on group on other column? for example: Test1|Test2|Test3|Test4|Test5 Test1|Test6|Test7|Test8|Test5 Test1|Test9|Test10|Test11|Test12 Test1|Test13|Test14|Test15|Test16 Test17|Test18|Test19|Test20|Test21 Test17|Test22|Test23|Test24|Test5 ... (2 Replies)
Discussion started by: yale_work
2 Replies

6. Shell Programming and Scripting

Remove Duplicate by considering multiple columns

hi friends, my input chr1 exon 35204 35266 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 42357 42473 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 45261 45404 gene_id "GOLGB1"; transcript_id "GOLGB1"; chr1 exon 50701 50778 gene_id "GOLGB1"; transcript_id "GOLGB1";... (2 Replies)
Discussion started by: jacobs.smith
2 Replies

7. Shell Programming and Scripting

Remove duplicate value based on two field $4 and $5

Hi All, i have input file like below... CA009156;20091003;M;AWBKCA72;123;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;; CA009156;20091003;M;AWBKCA72;321;;CANADIAN WESTERN BANK;EDMONTON;;2300, 10303, JASPER AVENUE;;T5J 3X6;; CA009156;20091003;M;AWBKCA72;231;;CANADIAN... (2 Replies)
Discussion started by: mohan sharma
2 Replies

8. Shell Programming and Scripting

How To Remove Duplicate Based on the Value?

Hi , Some time i got duplicated value in my files , bundle_identifier= B Sometext=ABC bundle_identifier= A bundle_unit=500 Sometext123=ABCD bundle_unit=400 i need to check if there is a duplicated values or not if yes , i need to check if the value is A or B when Bundle_Identified ,... (2 Replies)
Discussion started by: OTNA
2 Replies

9. Shell Programming and Scripting

Remove columns with duplicate entries

I have a 13gb file. It has the following columns: The 3rd column is basically correlation values. I want to delete those rows which are repeated between the columns: A B 0.04 B C 0.56 B B 1 A A 1 C D 1 C C 1 Desired Output: (preferably in a .csv format A,B,0.04 B,C,0.56 C,D,1... (3 Replies)
Discussion started by: Sanchari
3 Replies

10. Shell Programming and Scripting

Remove duplicate lines from file based on fields

Dear community, I have to remove duplicate lines from a file contains a very big ammount of rows (milions?) based on 1st and 3rd columns The data are like this: Region 23/11/2014 09:11:36 41752 Medio 23/11/2014 03:11:38 4132 Info 23/11/2014 05:11:09 4323... (2 Replies)
Discussion started by: Lord Spectre
2 Replies
MAILQ(1)						      General Commands Manual							  MAILQ(1)

NAME
mailq - print the mail queue SYNOPSIS
mailq [-Ac] [-q...] [-v] DESCRIPTION
Mailq prints a summary of the mail messages queued for future delivery. The first line printed for each message shows the internal identifier used on this host for the message with a possible status character, the size of the message in bytes, the date and time the message was accepted into the queue, and the envelope sender of the message. The second line shows the error message that caused this message to be retained in the queue; it will not be present if the message is being processed for the first time. The status characters are either * to indicate the job is being processed; X to indicate that the load is too high to process the job; and - to indicate that the job is too young to process. The following lines show message recipients, one per line. Mailq is identical to ``sendmail -bp''. The relevant options are as follows: -Ac Show the mail submission queue specified in /etc/mail/submit.cf instead of the MTA queue specified in /etc/mail/sendmail.cf. -qL Show the "lost" items in the mail queue instead of the normal queue items. -qQ Show the quarantined items in the mail queue instead of the normal queue items. -q[!]I substr Limit processed jobs to those containing substr as a substring of the queue id or not when ! is specified. -q[!]Q substr Limit processed jobs to quarantined jobs containing substr as a substring of the quarantine reason or not when ! is specified. -q[!]R substr Limit processed jobs to those containing substr as a substring of one of the recipients or not when ! is specified. -q[!]S substr Limit processed jobs to those containing substr as a substring of the sender or not when ! is specified. -v Print verbose information. This adds the priority of the message and a single character indicator (``+'' or blank) indicating whether a warning message has been sent on the first line of the message. Additionally, extra lines may be intermixed with the recipients indicating the ``controlling user'' information; this shows who will own any programs that are executed on behalf of this message and the name of the alias this command expanded from, if any. Moreover, status messages for each recipient are printed if available. Several sendmail.cf options influence the behavior of the mailq utility: The number of items printed per queue group is restricted by MaxQueueRunSize if that value is set. The status character * is not printed for some values of QueueSortOrder, e.g., filename, random, modification, and none, unless a -q option is used to limit the processed jobs. The mailq utility exits 0 on success, and >0 if an error occurs. SEE ALSO
sendmail(8) HISTORY
The mailq command appeared in 4.0BSD. $Date: 2013-11-22 20:51:55 $ MAILQ(1)
All times are GMT -4. The time now is 11:02 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy