Sponsored Content
Top Forums Shell Programming and Scripting how to delete duplicate rows based on last column Post 302347194 by panyam on Tuesday 25th of August 2009 05:57:05 AM
Old 08-25-2009
something like this :
Code:
awk '{ va=$NF;$NF=" "; if ($0 in a) { if (va > a[$0]){a[$0]=va}} else {a[$0]=va}} END { for ( i in a ) print i" "a[i] }'  file_name.txt

need to check further as the order of the elements in associative array is not the same.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to delete duplicate rows in a file

I have a file content like below. "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","","" "0000000","ABLNCYI","BOTH",1049,2058,"XYZ","5711002","","Y","","","","","","","",""... (5 Replies)
Discussion started by: vamshikrishnab
5 Replies

2. UNIX for Dummies Questions & Answers

Remove duplicate rows of a file based on a value of a column

Hi, I am processing a file and would like to delete duplicate records as indicated by one of its column. e.g. COL1 COL2 COL3 A 1234 1234 B 3k32 2322 C Xk32 TTT A NEW XX22 B 3k32 ... (7 Replies)
Discussion started by: risk_sly
7 Replies

3. UNIX for Dummies Questions & Answers

forming duplicate rows based on value of a key

if the key (A or B or ...others) has 4 in its 3rd column the 1st A row has to form 4 dupicates along with the all the values of A in 4th column (2.9, 3.8, 4.2) . Hope I explain the question clearly. Cheers Ruby input "A" 1 4 2.9 "A" 2 5 ... (7 Replies)
Discussion started by: ruby_sgp
7 Replies

4. Ubuntu

delete duplicate rows with awk files

Hi every body I have some text file with a lots of duplicate rows like this: 165.179.568.197 154.893.836.174 242.473.396.153 165.179.568.197 165.179.568.197 165.179.568.197 154.893.836.174 how can I delete the repeated rows? Thanks Saeideh (2 Replies)
Discussion started by: sashtari
2 Replies

5. UNIX for Dummies Questions & Answers

Remove duplicate rows when >10 based on single column value

Hello, I'm trying to delete duplicates when there are more than 10 duplicates, based on the value of the first column. e.g. a 1 a 2 a 3 b 1 c 1 gives b 1 c 1 but requires 11 duplicates before it deletes. Thanks for the help Video tutorial on how to use code tags in The UNIX... (11 Replies)
Discussion started by: informaticist
11 Replies

6. Shell Programming and Scripting

Delete duplicate rows

Hi, This is a followup to my earlier post him mno klm 20 76 . + . klm_mango unix_00000001; alp fdc klm 123 456 . + . klm_mango unix_0000103; her tkr klm 415 439 . + . klm_mango unix_00001043; abc tvr klm 20 76 . + . klm_mango unix_00000001; abc def klm 83 84 . + . klm_mango... (5 Replies)
Discussion started by: jacobs.smith
5 Replies

7. UNIX for Dummies Questions & Answers

merging rows into new file based on rows and first column

I have 2 files, file01= 7 columns, row unknown (but few) file02= 7 columns, row unknown (but many) now I want to create an output with the first field that is shared in both of them and then subtract the results from the rest of the fields and print there e.g. file 01 James|0|50|25|10|50|30... (1 Reply)
Discussion started by: A-V
1 Replies

8. Shell Programming and Scripting

awk to sum a column based on duplicate strings in another column and show split totals

Hi, I have a similar input format- A_1 2 B_0 4 A_1 1 B_2 5 A_4 1 and looking to print in this output format with headers. can you suggest in awk?awk because i am doing some pattern matching from parent file to print column 1 of my input using awk already.Thanks! letter number_of_letters... (5 Replies)
Discussion started by: prashob123
5 Replies

9. Shell Programming and Scripting

Remove duplicate rows based on one column

Dear members, I need to filter a file based on the 8th column (that is id), and does not mather the other columns, because I want just one id (1 line of each id) and remove the duplicates lines based on this id (8th column), and does not matter wich duplicate will be removed. example of my file... (3 Replies)
Discussion started by: clarissab
3 Replies

10. Shell Programming and Scripting

Extract and exclude rows based on duplicate values

Hello I have a file like this: > cat examplefile ghi|NN603762|eee mno|NN607265|ttt pqr|NN613879|yyy stu|NN615002|uuu jkl|NN607265|rrr vwx|NN615002|iii yzA|NN618555|ooo def|NN190486|www BCD|NN628717|ppp abc|NN190486|qqq EFG|NN628717|aaa HIJ|NN628717|sss > I can sort the file by... (5 Replies)
Discussion started by: CHoggarth
5 Replies
DB2_COLUMNS(3)								 1							    DB2_COLUMNS(3)

db2_columns - Returns a result set listing the columns and associated metadata for a table

SYNOPSIS
resource db2_columns (resource $connection, [string $qualifier], [string $schema], [string $table-name], [string $column-name]) DESCRIPTION
Returns a result set listing the columns and associated metadata for a table. PARAMETERS
o $connection - A valid connection to an IBM DB2, Cloudscape, or Apache Derby database. o $qualifier - A qualifier for DB2 databases running on OS/390 or z/OS servers. For other databases, pass NULL or an empty string. o $schema - The schema which contains the tables. To match all schemas, pass '%'. o $table-name - The name of the table or view. To match all tables in the database, pass NULL or an empty string. o $column-name - The name of the column. To match all columns in the table, pass NULL or an empty string. RETURN VALUES
Returns a statement resource with a result set containing rows describing the columns matching the specified parameters. The rows are com- posed of the following columns: +------------------+---------------------------------------------------+ | Column name | | | | | | | Description | | | | +------------------+---------------------------------------------------+ | TABLE_CAT | | | | | | | Name of the catalog. The value is NULL if this | | | table does not have catalogs. | | | | | TABLE_SCHEM | | | | | | | Name of the schema. | | | | | TABLE_NAME | | | | | | | Name of the table or view. | | | | | COLUMN_NAME | | | | | | | Name of the column. | | | | | DATA_TYPE | | | | | | | The SQL data type for the column represented as | | | an integer value. | | | | | TYPE_NAME | | | | | | | A string representing the data type for the col- | | | umn. | | | | | COLUMN_SIZE | | | | | | | An integer value representing the size of the | | | column. | | | | | BUFFER_LENGTH | | | | | | | Maximum number of bytes necessary to store data | | | from this column. | | | | | DECIMAL_DIGITS | | | | | | | The scale of the column, or NULL where scale is | | | not applicable. | | | | | NUM_PREC_RADIX | | | | | | | An integer value of either 10 (representing an | | | exact numeric data type), 2 (representing an | | | approximate numeric data type), or NULL (repre- | | | senting a data type for which radix is not appli- | | | cable). | | | | | NULLABLE | | | | | | | An integer value representing whether the column | | | is nullable or not. | | | | | REMARKS | | | | | | | Description of the column. | | | | | COLUMN_DEF | | | | | | | Default value for the column. | | | | | SQL_DATA_TYPE | | | | | | | An integer value representing the size of the | | | column. | | | | |SQL_DATETIME_SUB | | | | | | | Returns an integer value representing a datetime | | | subtype code, or NULL for SQL data types to which | | | this does not apply. | | | | |CHAR_OCTET_LENGTH | | | | | | | Maximum length in octets for a character data | | | type column, which matches COLUMN_SIZE for sin- | | | gle-byte character set data, or NULL for non- | | | character data types. | | | | |ORDINAL_POSITION | | | | | | | The 1-indexed position of the column in the ta- | | | ble. | | | | | IS_NULLABLE | | | | | | | A string value where 'YES' means that the column | | | is nullable and 'NO' means that the column is not | | | nullable. | | | | +------------------+---------------------------------------------------+ SEE ALSO
db2_column_privileges(3), db2_foreign_keys(3), db2_primary_keys(3), db2_procedure_columns(3), db2_procedures(3), db2_special_columns(3), db2_statistics(3), db2_table_privileges(3), db2_tables(3). PHP Documentation Group DB2_COLUMNS(3)
All times are GMT -4. The time now is 02:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy