Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Select unique names while removing the duplicates from a column
Top Forums UNIX for Beginners Questions & Answers Select unique names while removing the duplicates from a column Post 303044290 by MadeInGermany on Tuesday 18th of February 2020 04:30:29 AM
Old 02-18-2020
Code:
awk -F, 'a[$1]++==0' filename

is quick and dirty because it stores an unnecessary integer value.
The full and efficient code is
Code:
awk -F, '!($1 in a) { a[$1]; print }' filename

That you can condense again to an implicit print
Code:
awk -F, '!(($1 in a) || a[$1])' filename

or
Code:
awk -F, '!($1 in a) && !a[$1]' filename

This User Gave Thanks to MadeInGermany For This Post:
RavinderSingh13
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

select unique values from duplicates in linux

I have values in the variable as so the for loop just fetches one by one params=$'$HEW_SRC_DATABASE_LIB\nprmAttunityUser\nprmAttunityPwd\nprmODBCDataSource\nprmLoadInd\nprmSrc_Lib_ATM\nprmODBCDataSource_ATM' and i have a grep command like this ret=`grep \$y $pf` ... (0 Replies)
Discussion started by: vee_789
0 Replies

2. Shell Programming and Scripting

Change unique file names into new unique filenames

I have 84 files with the following names splitseqs.1, spliseqs.2 etc. and I want to change the .number to a unique filename. E.g. change splitseqs.1 into splitseqs.7114_1#24 and change spliseqs.2 into splitseqs.7067_2#4 So all the current file names are unique, so are the new file names.... (1 Reply)
Discussion started by: avonm
1 Replies

3. Shell Programming and Scripting

Select Unique Value

HOW CAN I SELECT AN UNIQUE STRING FROM A FIELD? ACTUALLY I WANT TO PRINT RECORDS THAT 2ND FIELD OF THAT HAVE ONE CHARACTER AND IT MUST BE "P" AWK '$2~"" {PRINT $0}' IN > OUTBUT THIS CODE PRINT ALL RECORDS WHICH 2ND FIELDS OF THEM START WITH "P" AND MAY CONTAINS ANOTHER CHARACTER! (1 Reply)
Discussion started by: saeed.soltani
1 Replies

4. Shell Programming and Scripting

Print unique names in each row of a specific column using awk

Is it possible to remove redundant names in the 4th column? input cqWE 100 200 singapore;singapore AZO 300 400 brazil;america;germany;ireland;germany .... .... output cqWE 100 200 singapore AZO 300 400 brazil;america;germany;ireland (4 Replies)
Discussion started by: quincyjones
4 Replies

5. Shell Programming and Scripting

Print unique names in a specific column using awk

Is it possible to modify file like this. 1. Remove all the duplicate names in a define column i.e 4th col 2. Count the no.of unique names separated by ";" and print as a 5th col thanx in advance!! Q input c1 30 3 Eh2 c10 96 3 Frp c41 396 3 Ua5;Lop;Kol;Kol c62 2 30 Fmp;Fmp;Fmp ... (5 Replies)
Discussion started by: quincyjones
5 Replies

6. Programming

Query to SELECT only Column Names that Contain a Specific String?

Hey Guys, I'm using SQuirreL SQL v3.5 GUI to fetch some data that I need for something I'm working on. I'm also using the IBM Informix Driver (*Version 3.5) to connect to the Database. What I want to do, if it's even possible, is to show all COLUMNS if they contain the word "Email". So in... (2 Replies)
Discussion started by: mrm5102
2 Replies

7. UNIX for Dummies Questions & Answers

Removing rows that contain non-unique column entry

Background: I have a file of thousands of potential SSR primers from Batch Primer 3. I can't use primers that will contain the same sequence ID or sequence as another primer. I have some basic shell scripting skills, but not enough to handle this. What you need to know: I need to remove the... (1 Reply)
Discussion started by: msatseqs
1 Replies

8. Shell Programming and Scripting

Removing duplicates on a single "column" (delimited file)

Hello ! I'm quite new to linux but haven't found a script to do this task, unfortunately my knowledge is quite limited on shellscripts... Could you guys help me removing the duplicate lines of a file, based only on a single "column"? For example: M202034357;01/2008;J30RJ021;Ciclo 01... (4 Replies)
Discussion started by: Rufinofr
4 Replies

9. Shell Programming and Scripting

Count occurrence of column one unique value having unique second column value

Hello Team, I need your help on the following: My input file a.txt is as below: 3330690|373846|108471 3330690|373846|108471 0640829|459725|100001 0640829|459725|100001 3330690|373847|108471 Here row 1 and row 2 of column 1 are identical but corresponding column 2 value are... (4 Replies)
Discussion started by: angshuman
4 Replies

10. Shell Programming and Scripting

Duplicates and unique segregation

Hi All, I have multiple files and i need to segregate unique and duplicates into files. Eg: /source/ -- path abc_12092016.csv abc_11092016.csv abc_12092016.csv ID,NAME,NUMBER 1,XYZ,1234 2,SDF,3456 1,XYZ,1234 abc_11092016.csv 4,RTY,7890 6,WER,5678 8,YUI,0987 6,WER,5678 in the... (1 Reply)
Discussion started by: ajayr1982
1 Replies

LEARN ABOUT PHP

db2_special_columns

DB2_SPECIAL_COLUMNS(3)							 1						    DB2_SPECIAL_COLUMNS(3)

db2_special_columns - Returns a result set listing the unique row identifier columns for a table

SYNOPSIS

       resource db2_special_columns (resource  $connection, string  $qualifier, string	$schema, string  $table_name, int  $scope)

DESCRIPTION

	Returns a result set listing the unique row identifier columns for a table.

PARAMETERS

	      o $connection
		- A valid connection to an IBM DB2, Cloudscape, or Apache Derby database.

	      o $qualifier
		- A qualifier for DB2 databases running on OS/390 or z/OS servers. For other databases, pass NULL or an empty string.

	      o $schema
		- The schema which contains the tables.

	      o $table_name
		- The name of the table.

	      o $scope
		-  Integer  value representing the minimum duration for which the unique row identifier is valid. This can be one of the following
		values:

	      +--------------+--------------------------------------+---+
	      |Integer value |					    |	|
	      | 	     |					    |	|
	      | 	     |		   SQL constant 	    |	|
	      | 	     |					    |	|
	      | 	     |		   Description		    |	|
	      | 	     |					    |	|
	      +--------------+--------------------------------------+---+
	      |      0	     |					    |	|
	      | 	     |					    |	|
	      | 	     |		 SQL_SCOPE_CURROW	    |	|
	      | 	     |					    |	|
	      | 	     | Row identifier is valid	only  while |	|
	      | 	     | the cursor is positioned on the row. |	|
	      | 	     |					    |	|
	      |      1	     |					    |	|
	      | 	     |					    |	|
	      | 	     |	      SQL_SCOPE_TRANSACTION	    |	|
	      | 	     |					    |	|
	      | 	     | Row  identifier	is  valid  for	the |	|
	      | 	     | duration of the transaction.	    |	|
	      | 	     |					    |	|
	      |      2	     |					    |	|
	      | 	     |					    |	|
	      | 	     |		SQL_SCOPE_SESSION	    |	|
	      | 	     |					    |	|
	      | 	     | Row  identifier	is  valid  for	the |	|
	      | 	     | duration of the connection.	    |	|
	      | 	     |					    |	|
	      +--------------+--------------------------------------+---+
RETURN VALUES

	Returns  a  statement resource with a result set containing rows with unique row identifier information for a table. The rows are composed
       of the following columns:

       +------------+---------------------------------------------------+
       |Column name |							|
       |	    |							|
       |	    |			 Description			|
       |	    |							|
       +------------+---------------------------------------------------+
       |   SCOPE    |							|
       |	    |							|
       |	    |							|
       |	    |							|
       |	    |	     box, tab (|); c | c | c |	.   T{	Integer |
       |	    |	     value					|
       |	    |							|
       |	    |			SQL constant			|
       |	    |							|
       |	    |			 Description			|
       |	    |							|
       +------------+---------------------------------------------------+
       |     0	    |							|
       |	    |							|
       |	    |		      SQL_SCOPE_CURROW			|
       |	    |							|
       |	    | Row  identifier is valid only while the cursor is |
       |	    | positioned on the row.				|
       |	    |							|
       |     1	    |							|
       |	    |							|
       |	    |		    SQL_SCOPE_TRANSACTION		|
       |	    |							|
       |	    | Row identifier is valid for the duration	of  the |
       |	    | transaction.					|
       |	    |							|
       |     2	    |							|
       |	    |							|
       |	    |		      SQL_SCOPE_SESSION 		|
       |	    |							|
       |	    | Row  identifier  is valid for the duration of the |
       |	    | connection.					|
       |	    |							|
       +------------+---------------------------------------------------+
       T} T{ COLUMN_NAME

       T} |T{ Name of the unique column.

       T} T{ DATA_TYPE

       T} |T{ SQL data type for the column.

       T} T{ TYPE_NAME

       T} |T{ Character string representation of the SQL data type for the column.

       T} T{ COLUMN_SIZE

       T} |T{ An integer value representing the size of the column.

       T} T{ BUFFER_LENGTH

       T} |T{
	Maximum number of bytes necessary to store data from this column.

       T} T{ DECIMAL_DIGITS

       T} |T{
	The scale of the column, or NULL where scale is not applicable.

       T} T{ NUM_PREC_RADIX

       T} |T{
	An integer value of either 10 (representing an exact numeric data type), 2 (representing an approximate numeric data type), or NULL  (rep-
       resenting a data type for which radix is not applicable).

       T} T{ PSEUDO_COLUMN

       T} |T{ Always returns 1.

       T}

SEE ALSO

       db2_column_privileges(3),  db2_columns(3),  db2_foreign_keys(3), db2_primary_keys(3), db2_procedure_columns(3), db2_procedures(3), db2_sta-
       tistics(3), db2_table_privileges(3), db2_tables(3).

PHP Documentation Group 												    DB2_SPECIAL_COLUMNS(3)
All times are GMT -4. The time now is 01:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy