Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Select unique names while removing the duplicates from a column Post 303044290 by MadeInGermany on Tuesday 18th of February 2020 04:30:29 AM
Old 02-18-2020
Code:
awk -F, 'a[$1]++==0' filename

is quick and dirty because it stores an unnecessary integer value.
The full and efficient code is
Code:
awk -F, '!($1 in a) { a[$1]; print }' filename

That you can condense again to an implicit print
Code:
awk -F, '!(($1 in a) || a[$1])' filename

or
Code:
awk -F, '!($1 in a) && !a[$1]' filename

This User Gave Thanks to MadeInGermany For This Post:
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

select unique values from duplicates in linux

I have values in the variable as so the for loop just fetches one by one params=$'$HEW_SRC_DATABASE_LIB\nprmAttunityUser\nprmAttunityPwd\nprmODBCDataSource\nprmLoadInd\nprmSrc_Lib_ATM\nprmODBCDataSource_ATM' and i have a grep command like this ret=`grep \$y $pf` ... (0 Replies)
Discussion started by: vee_789
0 Replies

2. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

3. Shell Programming and Scripting

Select Unique Value

HOW CAN I SELECT AN UNIQUE STRING FROM A FIELD? ACTUALLY I WANT TO PRINT RECORDS THAT 2ND FIELD OF THAT HAVE ONE CHARACTER AND IT MUST BE "P" AWK '$2~"" {PRINT $0}' IN > OUTBUT THIS CODE PRINT ALL RECORDS WHICH 2ND FIELDS OF THEM START WITH "P" AND MAY CONTAINS ANOTHER CHARACTER! (1 Reply)
Discussion started by: saeed.soltani
1 Replies

4. Shell Programming and Scripting

Print unique names in each row of a specific column using awk

Is it possible to remove redundant names in the 4th column? input cqWE 100 200 singapore;singapore AZO 300 400 brazil;america;germany;ireland;germany .... .... output cqWE 100 200 singapore AZO 300 400 brazil;america;germany;ireland (4 Replies)
Discussion started by: quincyjones
4 Replies

5. Shell Programming and Scripting

Print unique names in a specific column using awk

Is it possible to modify file like this. 1. Remove all the duplicate names in a define column i.e 4th col 2. Count the no.of unique names separated by ";" and print as a 5th col thanx in advance!! Q input c1 30 3 Eh2 c10 96 3 Frp c41 396 3 Ua5;Lop;Kol;Kol c62 2 30 Fmp;Fmp;Fmp ... (5 Replies)
Discussion started by: quincyjones
5 Replies

6. Programming

Query to SELECT only Column Names that Contain a Specific String?

Hey Guys, I'm using SQuirreL SQL v3.5 GUI to fetch some data that I need for something I'm working on. I'm also using the IBM Informix Driver (*Version 3.5) to connect to the Database. What I want to do, if it's even possible, is to show all COLUMNS if they contain the word "Email". So in... (2 Replies)
Discussion started by: mrm5102
2 Replies

7. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

8. Shell Programming and Scripting

Removing duplicates on a single "column" (delimited file)

Hello ! I'm quite new to linux but haven't found a script to do this task, unfortunately my knowledge is quite limited on shellscripts... Could you guys help me removing the duplicate lines of a file, based only on a single "column"? For example: M202034357;01/2008;J30RJ021;Ciclo 01... (4 Replies)
Discussion started by: Rufinofr
4 Replies

9. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

10. Shell Programming and Scripting

Duplicates and unique segregation

Hi All, I have multiple files and i need to segregate unique and duplicates into files. Eg: /source/ -- path abc_12092016.csv abc_11092016.csv abc_12092016.csv ID,NAME,NUMBER 1,XYZ,1234 2,SDF,3456 1,XYZ,1234 abc_11092016.csv 4,RTY,7890 6,WER,5678 8,YUI,0987 6,WER,5678 in the... (1 Reply)
Discussion started by: ajayr1982
1 Replies
DB2_STATISTICS(3)							 1							 DB2_STATISTICS(3)

db2_statistics - Returns a result set listing the index and statistics for a table

SYNOPSIS
resource db2_statistics (resource $connection, string $qualifier, string $schema, string $table-name, bool $unique) DESCRIPTION
Returns a result set listing the index and statistics for a table. PARAMETERS
o $connection - A valid connection to an IBM DB2, Cloudscape, or Apache Derby database. o $qualifier - A qualifier for DB2 databases running on OS/390 or z/OS servers. For other databases, pass NULL or an empty string. o $schema - The schema that contains the targeted table. If this parameter is NULL, the statistics and indexes are returned for the schema of the current user. o $table_name - The name of the table. o $unique - An integer value representing the type of index information to return. o $0 - Return only the information for unique indexes on the table. o $1 - Return the information for all indexes on the table. RETURN VALUES
Returns a statement resource with a result set containing rows describing the statistics and indexes for the base tables matching the specified parameters. The rows are composed of the following columns: +--------------+---------------------------------------------------+ | Column name | | | | | | | Description | | | | +--------------+---------------------------------------------------+ | TABLE_CAT | | | | | | | The catalog that contains the table. The value is | | | NULL if this table does not have catalogs. | | | | | TABLE_SCHEM | | | | | | | Name of the schema that contains the table. | | | | | TABLE_NAME | | | | | | | Name of the table. | | | | | NON_UNIQUE | | | | | | | An integer value representing whether the index | | | prohibits unique values, or whether the row rep- | | | resents statistics on the table itself: | | | | | | box, tab (|); c | c | . T{ Return value | | | | | | Parameter type | | | | +--------------+---------------------------------------------------+ |0 (SQL_FALSE) | | | | | | | The index allows duplicate values. | | | | |1 (SQL_TRUE) | | | | | | | The index values must be unique. | | | | | | | | NULL | | | | | | | This row is statistics information for the table | | | itself. | | | | +--------------+---------------------------------------------------+ T} T{ INDEX_QUALIFIER T} |T{ A string value representing the qualifier that would have to be prepended to INDEX_NAME to fully qualify the index. T} T{ INDEX_NAME T} |T{ A string representing the name of the index. T} T{ TYPE T} |T{ An integer value representing the type of information contained in this row of the result set: +------------------------+---------------------------------------------------+ | Return value | | | | | | | Parameter type | | | | +------------------------+---------------------------------------------------+ | 0 (SQL_TABLE_STAT) | | | | | | | The row contains statistics about the table | | | itself. | | | | |1 (SQL_INDEX_CLUSTERED) | | | | | | | The row contains information about a clustered | | | index. | | | | | 2 (SQL_INDEX_HASH) | | | | | | | The row contains information about a hashed | | | index. | | | | | 3 (SQL_INDEX_OTHER) | | | | | | | The row contains information about a type of | | | index that is neither clustered nor hashed. | | | | +------------------------+---------------------------------------------------+ T} T{ ORDINAL_POSITION T} |T{ The 1-indexed position of the column in the index. NULL if the row contains statistics information about the table itself. T} T{ COLUMN_NAME T} |T{ The name of the column in the index. NULL if the row contains statistics information about the table itself. T} T{ ASC_OR_DESC T} |T{ A if the column is sorted in ascending order, D if the column is sorted in descending order, NULL if the row contains statistics informa- tion about the table itself. T} T{ CARDINALITY T} |T{ If the row contains information about an index, this column contains an integer value representing the number of unique values in the index. If the row contains information about the table itself, this column contains an integer value representing the number of rows in the table. T} T{ PAGES T} |T{ If the row contains information about an index, this column contains an integer value representing the number of pages used to store the index. If the row contains information about the table itself, this column contains an integer value representing the number of pages used to store the table. T} T{ FILTER_CONDITION T} |T{ Always returns NULL. T} SEE ALSO
db2_column_privileges(3), db2_columns(3), db2_foreign_keys(3), db2_primary_keys(3), db2_procedure_columns(3), db2_procedures(3), db2_spe- cial_columns(3), db2_table_privileges(3), db2_tables(3). PHP Documentation Group DB2_STATISTICS(3)
All times are GMT -4. The time now is 08:08 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy