Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Remove duplicates in a dataframe (table) keeping all the different cells of just one of the columns
Top Forums UNIX for Beginners Questions & Answers Remove duplicates in a dataframe (table) keeping all the different cells of just one of the columns Post 303032096 by nezabudka on Monday 11th of March 2019 09:35:04 AM
Old 03-11-2019
Code:
awk '
($1 in A)       { if($2 > A[$1][2]) A[$1][2] = $2
                        A[$1][4] = A[$1][4] ";" $4
                        next
                }
                { for(n = split($0, M); n; n--) A[$1][n] = M[n]
                }
END             { for(i in A) {
                        for(j = 1; j <= NF; j++) printf "%s ",  A[i][j]
                                print ""
                        }
                }' file

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Deleting table cells in a script

I'd like to use sed or awk to do this but I'm weak on both along with RE. Looking for a way with sed or awk to count for the 7th table data within a table row and if the condition is met to delete "<td>and everything in between </td>". Since the table header start on a specific line each time, that... (15 Replies)
Discussion started by: phpfreak
15 Replies

2. Shell Programming and Scripting

Remove duplicates based on the two key columns

Hi All, I needs to fetch unique records based on a keycolumn(ie., first column1) and also I needs to get the records which are having max value on column2 in sorted manner... and duplicates have to store in another output file. Input : Input.txt 1234,0,x 1234,1,y 5678,10,z 9999,10,k... (7 Replies)
Discussion started by: kmsekhar
7 Replies

3. Shell Programming and Scripting

Search based on 1,2,4,5 columns and remove duplicates in the same file.

Hi, I am unable to search the duplicates in a file based on the 1st,2nd,4th,5th columns in a file and also remove the duplicates in the same file. Source filename: Filename.csv "1","ccc","information","5000","temp","concept","new" "1","ddd","information","6000","temp","concept","new"... (2 Replies)
Discussion started by: onesuri
2 Replies

4. UNIX for Dummies Questions & Answers

Two files; if cells match then copy over other columns

My current issue is dealing with two space delimited files. The first file has column 1 as the sample ID's, then columns 2 - n as the observations. The second file has column 1 as the sample ID's, column 2 as the mother ID's, column 3 as the father ID's, column 4 as the gender, and column 5... (3 Replies)
Discussion started by: Renyulb28
3 Replies

5. UNIX Desktop Questions & Answers

Using grep to remove cells instead of lines

I would like to use grep to remove certain strings from a text file but I can't use the grep -v option because it removes the whole line that includes the string whereas I just want to remove the string. How do I go about doing that? My input file: Magmas CEU rs12542019 CPNE1 RBM12 CEU... (1 Reply)
Discussion started by: evelibertine
1 Replies

6. Shell Programming and Scripting

CSV with commas in field values, remove duplicates, cut columns

Hi Description of input file I have: ------------------------- 1) CSV with double quotes for string fields. 2) Some string fields have Comma as part of field value. 3) Have Duplicate lines 4) Have 200 columns/fields 5) File size is more than 10GB Description of output file I need:... (4 Replies)
Discussion started by: krishnix
4 Replies

7. Shell Programming and Scripting

Remove Duplicates on multiple Key Columns and get the Latest Record from Date/Time Column

Hi Experts , we have a CDC file where we need to get the latest record of the Key columns Key Columns will be CDC_FLAG and SRC_PMTN_I and fetch the latest record from the CDC_PRCS_TS Can we do it with a single awk command. Please help.... (3 Replies)
Discussion started by: vijaykodukula
3 Replies

8. Shell Programming and Scripting

Remove duplicates by keeping the order intact

Hello friends, I have a file with duplicate lines. I could eliminate duplicate lines by running sort <file> |uniq >uniq_file and it works fine BUT it changes the order of the entries as it we did "sort". I need to remove duplicates and also need to keep the order/sequence of entries. I... (1 Reply)
Discussion started by: magnus29
1 Replies

9. UNIX for Beginners Questions & Answers

Merge cells in all rows of a HTML table dynamically.

Hello All, I have visited many pages in Unix.com and could find out one solution for merging the HTML cells in the 1st row. (Unable to post the complete URL as I should not as per website rules). But, however I try, I couldn't achieve this merging to happen for all other rows of HTML... (17 Replies)
Discussion started by: Mounika
17 Replies

10. UNIX for Beginners Questions & Answers

Sort and remove duplicates in directory based on first 5 columns:

I have /tmp dir with filename as: 010020001_S-FOR-Sort-SYEXC_20160229_2212101.marker 010020001_S-FOR-Sort-SYEXC_20160229_2212102.marker 010020001-S-XOR-Sort-SYEXC_20160229_2212104.marker 010020001-S-XOR-Sort-SYEXC_20160229_2212105.marker 010020001_S-ZOR-Sort-SYEXC_20160229_2212106.marker... (4 Replies)
Discussion started by: gnnsprapa
4 Replies

LEARN ABOUT PHP

odbc_specialcolumns

ODBC_SPECIALCOLUMNS(3)							 1						    ODBC_SPECIALCOLUMNS(3)

odbc_specialcolumns - Retrieves special columns

SYNOPSIS

       resource  odbc_specialcolumns  (resource   $connection_id, int  $type, string  $qualifier, string  $owner, string  $table, int  $scope, int
       $nullable)

DESCRIPTION

	Retrieves either the optimal set of columns that uniquely identifies a row in the table, or columns that are  automatically  updated  when
       any value in the row is updated by a transaction.

PARAMETERS

	      o $connection_id
		-The ODBC connection identifier, see odbc_connect(3) for details.

	      o $type
		-  When  the type argument is SQL_BEST_ROWID, odbc_specialcolumns(3) returns the column or columns that uniquely identify each row
		in the table.  When the type argument is SQL_ROWVER, odbc_specialcolumns(3) returns the column or columns in the specified  table,
		if any, that are automatically updated by the data source when any value in the row is updated by any transaction.

	      o $qualifier
		- The qualifier.

	      o $owner
		- The owner.

	      o $table
		- The table.

	      o $scope
		- The scope, which orders the result set.

	      o $nullable
		- The nullable option.

RETURN VALUES

	Returns an ODBC result identifier or FALSE on failure.

	The result set has the following columns:

	      oSCOPE

	      oCOLUMN_NAME

	      oDATA_TYPE

	      oTYPE_NAME

	      oPRECISION

	      oLENGTH

	      oSCALE

	      oPSEUDO_COLUMN

PHP Documentation Group 												    ODBC_SPECIALCOLUMNS(3)
All times are GMT -4. The time now is 10:33 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy