Sponsored Content
Top Forums Shell Programming and Scripting Remove rows with first 4 fields duplicated in awk Post 302568782 by ahamed101 on Friday 28th of October 2011 08:10:46 AM
Old 10-28-2011
Just extending Jayan's code... Requirement was first 4 fields not 4th field... Smilie
Code:
awk '!x[$1$2$3$4]++'  input_file

--ahamed
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ... (4 Replies)
Discussion started by: kiranmosarla
4 Replies

2. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486... (3 Replies)
Discussion started by: perl_beginner
3 Replies

3. Shell Programming and Scripting

awk to grep rows by multiple fields

Hello, I met a challenge to extract part of the table. I'd like to grep the first three matches based on field1 and field2. Input: D A 92.85 1315 83 11 D A 95.90 757 28 3 D A 94.38 480 20 7 D A 91.21 307 21 6 D A 94.26 244 ... (6 Replies)
Discussion started by: yifangt
6 Replies

4. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . . (7 Replies)
Discussion started by: the_simpsons
7 Replies

5. Shell Programming and Scripting

Delete duplicated fields in a line

Hi, I have files with this kind of format (separator is space): A1 B1 C1 D1 E1 F1 D1 C1 G1 H1 A2 B2 C2 D2 E2 F2 D2 C2 G2 H2 A3 B3 C3 D3 E3 F3 G3 D3 C3 H3 A4 B4 C4 D4 E4 F4 G4 D4 C4 H4 I want the output to be: A1 B1 E1 F1 G1 H1 A2 B2 E2 F2 G2 H2 A3 B3 E3 F3 G3 H3 A4 B4 E4 F4 G4... (12 Replies)
Discussion started by: Gr4wk
12 Replies

6. Shell Programming and Scripting

Removing duplicated first field rows

Hello, I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET". Data file: 345234|22.34|LST 546543|55.33|LST 793929|98.23|LST 793929|64.69|NET 149593|49.22|LST Desired output: 345234|22.34|LST 546543|55.33|LST... (2 Replies)
Discussion started by: palex
2 Replies

7. Shell Programming and Scripting

Merge files and remove duplicated rows

In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows. The file name in the folder will look like e.q: MissingData_2014-08-25_09-30-18.txt MissingData_2014-08-25_09-30-14.txt MissingData_2014-08-26_09-30-12.txt The content... (9 Replies)
Discussion started by: Bergans
9 Replies

8. Shell Programming and Scripting

Remove rows containing commas with awk

Hello everyone, I have a dataset that looks something like: 1 3 2 2 3 4,5 4 3:9 5 5,9 6 5:6 I need to remove the rows that contain a comma in the second column and I'm not sure how to go about this. Here is an attempt. awk 'BEGIN {FS=" "} { if ($2!==,) print }'Any help is appreciated. (5 Replies)
Discussion started by: Rabu
5 Replies

9. Shell Programming and Scripting

awk to remove range of fields

I am trying to cut a range of fields in awk. The below seems to work for removing field 50, but what is the correct syntax for removing a range ($50-$62). Thank you :). awk awk 'BEGIN{FS=OFS="\t"}{$50=""; gsub(/\t\t/,"\t")}1' test.vcf.hg19_multianno.txt > output.csv Maybe: awk... (6 Replies)
Discussion started by: cmccabe
6 Replies

10. Shell Programming and Scripting

awk to remove lines where field count is greather than 1 in two fields

I am trying to remove all the lines and spaces where the count in $4 or $5 is greater than 1 (more than 1 letter). The file and the output are tab-delimited. Thank you :). file X 5811530 . G C NLGN4X 17 10544696 . GA G MYH3 9 96439004 . C ... (1 Reply)
Discussion started by: cmccabe
1 Replies
IFX_FETCH_ROW(3)							 1							  IFX_FETCH_ROW(3)

ifx_fetch_row - Get row as an associative array

SYNOPSIS
array ifx_fetch_row (resource $result_id, [mixed $position]) DESCRIPTION
Fetches one row of data from the result associated with the specified result identifier. Subsequent calls to ifx_fetch_row(3) would return the next row in the result set, or FALSE if there are no more rows. PARAMETERS
o $result_id -$result_id is a valid resultid returned by ifx_query(3) or ifx_prepare(3) (select type queries only!). o $position - An optional parameter for a "fetch" operation on "scroll" cursors: NEXT, PREVIOUS, CURRENT, FIRST, LAST or a number. If you specify a number, an "absolute" row fetch is executed. This parameter is optional, and only valid for SCROLL cursors. RETURN VALUES
Returns an associative array that corresponds to the fetched row, or FALSE if there are no more rows. Blob columns are returned as integer blob id values for use in ifx_get_blob(3) unless you have used ifx_textasvarchar(1) or ifx_byteasvar- char(1), in which case blobs are returned as string values. EXAMPLES
Example #1 Informix fetch rows <?php $rid = ifx_prepare ("select * from emp where name like " . $name, $connid, IFX_SCROLL); if (! $rid) { /* ... error ... */ } $rowcount = ifx_affected_rows($rid); if ($rowcount > 1000) { printf ("Too many rows in result set (%d) <br />", $rowcount); die ("Please restrict your query<br /> "); } if (! ifx_do ($rid)) { /* ... error ... */ } $row = ifx_fetch_row ($rid, "NEXT"); while (is_array($row)) { for (reset($row); $fieldname=key($row); next($row)) { $fieldvalue = $row[$fieldname]; printf ("%s = %s,", $fieldname, $fieldvalue); } printf(" <br />"); $row = ifx_fetch_row($rid, "NEXT"); } ifx_free_result ($rid); ?> PHP Documentation Group IFX_FETCH_ROW(3)
All times are GMT -4. The time now is 03:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy