Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Select unique names while removing the duplicates from a column Post 303044284 by jim mcnamara on Monday 17th of February 2020 11:38:50 PM
Old 02-18-2020
I think you need to specify a field separator as a comma.

Code:
Owner@Owner-PC ~
$ awk -F, '!a[$1]++' filename
ENSG00000003137,ENST00000001146
ENSG00000003402,ENST00000309955


Owner@Owner-PC ~
$ awk  '!a[$1]++' filename
ENSG00000003137,ENST00000001146
ENSG00000003137,ENST00000412253
ENSG00000003402,ENST00000309955
ENSG00000003402,ENST00000443227
ENSG00000003402,ENST00000341222

I used the sample data
These 2 Users Gave Thanks to jim mcnamara For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

select unique values from duplicates in linux

I have values in the variable as so the for loop just fetches one by one params=$'$HEW_SRC_DATABASE_LIB\nprmAttunityUser\nprmAttunityPwd\nprmODBCDataSource\nprmLoadInd\nprmSrc_Lib_ATM\nprmODBCDataSource_ATM' and i have a grep command like this ret=`grep \$y $pf` ... (0 Replies)
Discussion started by: vee_789
0 Replies

2. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

3. Shell Programming and Scripting

Select Unique Value

HOW CAN I SELECT AN UNIQUE STRING FROM A FIELD? ACTUALLY I WANT TO PRINT RECORDS THAT 2ND FIELD OF THAT HAVE ONE CHARACTER AND IT MUST BE "P" AWK '$2~"" {PRINT $0}' IN > OUTBUT THIS CODE PRINT ALL RECORDS WHICH 2ND FIELDS OF THEM START WITH "P" AND MAY CONTAINS ANOTHER CHARACTER! (1 Reply)
Discussion started by: saeed.soltani
1 Replies

4. Shell Programming and Scripting

Print unique names in each row of a specific column using awk

Is it possible to remove redundant names in the 4th column? input cqWE 100 200 singapore;singapore AZO 300 400 brazil;america;germany;ireland;germany .... .... output cqWE 100 200 singapore AZO 300 400 brazil;america;germany;ireland (4 Replies)
Discussion started by: quincyjones
4 Replies

5. Shell Programming and Scripting

Print unique names in a specific column using awk

Is it possible to modify file like this. 1. Remove all the duplicate names in a define column i.e 4th col 2. Count the no.of unique names separated by ";" and print as a 5th col thanx in advance!! Q input c1 30 3 Eh2 c10 96 3 Frp c41 396 3 Ua5;Lop;Kol;Kol c62 2 30 Fmp;Fmp;Fmp ... (5 Replies)
Discussion started by: quincyjones
5 Replies

6. Programming

Query to SELECT only Column Names that Contain a Specific String?

Hey Guys, I'm using SQuirreL SQL v3.5 GUI to fetch some data that I need for something I'm working on. I'm also using the IBM Informix Driver (*Version 3.5) to connect to the Database. What I want to do, if it's even possible, is to show all COLUMNS if they contain the word "Email". So in... (2 Replies)
Discussion started by: mrm5102
2 Replies

7. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

8. Shell Programming and Scripting

Removing duplicates on a single "column" (delimited file)

Hello ! I'm quite new to linux but haven't found a script to do this task, unfortunately my knowledge is quite limited on shellscripts... Could you guys help me removing the duplicate lines of a file, based only on a single "column"? For example: M202034357;01/2008;J30RJ021;Ciclo 01... (4 Replies)
Discussion started by: Rufinofr
4 Replies

9. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

10. Shell Programming and Scripting

Duplicates and unique segregation

Hi All, I have multiple files and i need to segregate unique and duplicates into files. Eg: /source/ -- path abc_12092016.csv abc_11092016.csv abc_12092016.csv ID,NAME,NUMBER 1,XYZ,1234 2,SDF,3456 1,XYZ,1234 abc_11092016.csv 4,RTY,7890 6,WER,5678 8,YUI,0987 6,WER,5678 in the... (1 Reply)
Discussion started by: ajayr1982
1 Replies
PTS_LISTENTRIES(1)					       AFS Command Reference						PTS_LISTENTRIES(1)

NAME
pts_listentries - Displays all users or groups in the Protection Database SYNOPSIS
pts listentries [-users] [-groups] [-cell <cell name>] [-noauth] [-localauth] [-force] [-help] pts liste [-u] [-g] [-c <cell name>] [-n] [-l] [-f] [-h] DESCRIPTION
The pts listentries command displays the name and AFS ID of all Protection Database entries of the indicated type. It also displays the AFS ID of each entry's owner and creator. To display all user and machine entries, either include the -users flag or omit both it and the -groups flag. To display all group entries, include the -groups flag. To display all entries, provide both flags. OPTIONS
-users Displays user and machine entries. -groups Displays group entries. -cell <cell name> Names the cell in which to run the command. For more details, see pts(1). -force Enables the command to continue executing as far as possible when errors or other problems occur, rather than halting execution at the first error. -help Prints the online help for this command. All other valid options are ignored. -localauth Constructs a server ticket using a key from the local /etc/openafs/server/KeyFile file. Do not combine this flag with the -cell or -noauth options. For more details, see pts(1). -noauth Assigns the unprivileged identity anonymous to the issuer. For more details, see pts(1). OUTPUT
The output includes a line for each entry, with information in four columns that have the following headers: Name The entry's name. ID The entry's AFS ID (AFS UID for a user or machine, negative AFS GID for a group). Owner The AFS ID of the user or group that owns the entry. Creator The AFS ID of the user who created the entry (the system:administrators group is listed as the creator of the entry for "anonymous" and the system groups, but it is not otherwise possible for a group to create groups). In general, the entries appear in the order in which they were created. EXAMPLES
The following example displays both user and group entries. % pts listentries -users -groups Name ID Owner Creator system:administrators -204 -204 -204 system:anyuser -101 -204 -204 system:authuser -102 -204 -204 anonymous 32766 -204 -204 admin 1 -204 32766 pat 100 -204 1 smith 101 -204 1 pat:friends -206 100 100 staff -207 -204 1 PRIVILEGE REQUIRED
The issuer must belong to the system:administrators group. SEE ALSO
pts(1), pts_creategroup(1), pts_createuser(1), pts_examine(1) COPYRIGHT
IBM Corporation 2000. <http://www.ibm.com/> All Rights Reserved. This documentation is covered by the IBM Public License Version 1.0. It was converted from HTML to POD by software written by Chas Williams and Russ Allbery, based on work by Alf Wachsmann and Elizabeth Cassell. OpenAFS 2014-04-08 PTS_LISTENTRIES(1)
All times are GMT -4. The time now is 04:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy