Remove rows with first 4 fields duplicated in awk Post: 302568733

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk script to remove duplicate rows in line

i have the long file more than one ns and www and mx in the line like . i need the first ns record and first www and first mx from line . the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution. ...

2. Shell Programming and Scripting

Help with remove duplicated content

Input file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hcmv-UL70-3p hsa-4486 hcms-US25 hsa-360-5 hcms-US25 hsa-4 hcms-US25 hsa-458 hcms-US25 hsa-44812 . . Desired Output file: hcmv-US25-2-3p hsa-3160-5 hcmv-US33 hsa-47 hcmv-UL70-3p hsa-4508 hsa-4486...

3. Shell Programming and Scripting

awk to grep rows by multiple fields

Hello, I met a challenge to extract part of the table. I'd like to grep the first three matches based on field1 and field2. Input: D A 92.85 1315 83 11 D A 95.90 757 28 3 D A 94.38 480 20 7 D A 91.21 307 21 6 D A 94.26 244 ...

4. Shell Programming and Scripting

How to remove duplicated lines?

Hi, if i have a file like this: Query=1 a a b c c c d Query=2 b b b c c e . . .

5. Shell Programming and Scripting

Delete duplicated fields in a line

Hi, I have files with this kind of format (separator is space): A1 B1 C1 D1 E1 F1 D1 C1 G1 H1 A2 B2 C2 D2 E2 F2 D2 C2 G2 H2 A3 B3 C3 D3 E3 F3 G3 D3 C3 H3 A4 B4 C4 D4 E4 F4 G4 D4 C4 H4 I want the output to be: A1 B1 E1 F1 G1 H1 A2 B2 E2 F2 G2 H2 A3 B3 E3 F3 G3 H3 A4 B4 E4 F4 G4...

6. Shell Programming and Scripting

Removing duplicated first field rows

Hello, I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET". Data file: 345234|22.34|LST 546543|55.33|LST 793929|98.23|LST 793929|64.69|NET 149593|49.22|LST Desired output: 345234|22.34|LST 546543|55.33|LST...

7. Shell Programming and Scripting

Merge files and remove duplicated rows

In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows. The file name in the folder will look like e.q: MissingData_2014-08-25_09-30-18.txt MissingData_2014-08-25_09-30-14.txt MissingData_2014-08-26_09-30-12.txt The content...

8. Shell Programming and Scripting

Remove rows containing commas with awk

Hello everyone, I have a dataset that looks something like: 1 3 2 2 3 4,5 4 3:9 5 5,9 6 5:6 I need to remove the rows that contain a comma in the second column and I'm not sure how to go about this. Here is an attempt. awk 'BEGIN {FS=" "} { if ($2!==,) print }'Any help is appreciated.

9. Shell Programming and Scripting

awk to remove range of fields

I am trying to cut a range of fields in awk. The below seems to work for removing field 50, but what is the correct syntax for removing a range ($50-$62). Thank you :). awk awk 'BEGIN{FS=OFS="\t"}{$50=""; gsub(/\t\t/,"\t")}1' test.vcf.hg19_multianno.txt > output.csv Maybe: awk...

10. Shell Programming and Scripting

awk to remove lines where field count is greather than 1 in two fields

I am trying to remove all the lines and spaces where the count in $4 or $5 is greater than 1 (more than 1 letter). The file and the output are tab-delimited. Thank you :). file X 5811530 . G C NLGN4X 17 10544696 . GA G MYH3 9 96439004 . C ...

LEARN ABOUT PHP

filesystemiterator

FILESYSTEMITERATOR(3)							 1						     FILESYSTEMITERATOR(3)

The FilesystemIterator class

INTRODUCTION

	The Filesystem iterator

CLASS SYNOPSIS

       FilesystemIterator

       FilesystemIteratorextends
       DirectoryIteratorSeekableIterator

       Constants

       o const integer$FilesystemIterator::CURRENT_AS_PATHNAME32

       o const integer$FilesystemIterator::CURRENT_AS_FILEINFO0

       o const integer$FilesystemIterator::CURRENT_AS_SELF16

       o const integer$FilesystemIterator::CURRENT_MODE_MASK240

       o const integer$FilesystemIterator::KEY_AS_PATHNAME0

       o const integer$FilesystemIterator::KEY_AS_FILENAME256

       o const integer$FilesystemIterator::FOLLOW_SYMLINKS512

       o const integer$FilesystemIterator::KEY_MODE_MASK3840

       o const integer$FilesystemIterator::NEW_CURRENT_AND_KEY256

       o const integer$FilesystemIterator::SKIP_DOTS4096

       o const integer$FilesystemIterator::UNIX_PATHS8192

       Methods

       o public  FilesystemIterator::__construct  (string   $path,  [int   $flags = FilesystemIterator::KEY_AS_PATHNAME | FilesystemIterator::CUR-
	 RENT_AS_FILEINFO | FilesystemIterator::SKIP_DOTS])

       o public mixed FilesystemIterator::current (void  )

       o public int FilesystemIterator::getFlags (void	)

       o public string FilesystemIterator::key (void  )

       o public void FilesystemIterator::next (void  )

       o public void FilesystemIterator::rewind (void  )

       o public void FilesystemIterator::setFlags ([int  $flags])

       Inherited methods

       o public DirectoryIterator DirectoryIterator::current (void  )

       o public int DirectoryIterator::getATime (void  )

       o public string DirectoryIterator::getBasename ([string	$suffix])

       o public int DirectoryIterator::getCTime (void  )

       o public string DirectoryIterator::getExtension (void  )

       o public string DirectoryIterator::getFilename (void  )

       o public int DirectoryIterator::getGroup (void  )

       o public int DirectoryIterator::getInode (void  )

       o public int DirectoryIterator::getMTime (void  )

       o public int DirectoryIterator::getOwner (void  )

       o public string DirectoryIterator::getPath (void  )

       o public string DirectoryIterator::getPathname (void  )

       o public int DirectoryIterator::getPerms (void  )

       o public int DirectoryIterator::getSize (void  )

       o public string DirectoryIterator::getType (void  )

       o public bool DirectoryIterator::isDir (void  )

       o public bool DirectoryIterator::isDot (void  )

       o public bool DirectoryIterator::isExecutable (void  )

       o public bool DirectoryIterator::isFile (void  )

       o public bool DirectoryIterator::isLink (void  )

       o public bool DirectoryIterator::isReadable (void  )

       o public bool DirectoryIterator::isWritable (void  )

       o public string DirectoryIterator::key (void  )

       o public void DirectoryIterator::next (void  )

       o public void DirectoryIterator::rewind (void  )

       o public void DirectoryIterator::seek (int  $position)

       o public string DirectoryIterator::__toString (void  )

       o public bool DirectoryIterator::valid (void  )

PREDEFINED CONSTANTS

	      o FilesystemIterator::CURRENT_AS_PATHNAME -Makes FilesystemIterator::current return the pathname.

	      o FilesystemIterator::CURRENT_AS_FILEINFO -Makes FilesystemIterator::current return an SplFileInfo instance.

	      o FilesystemIterator::CURRENT_AS_SELF -Makes FilesystemIterator::current return $this (the FilesystemIterator).

	      o FilesystemIterator::CURRENT_MODE_MASK -Masks FilesystemIterator::current

	      o FilesystemIterator::KEY_AS_PATHNAME -Makes FilesystemIterator::key return the pathname.

	      o FilesystemIterator::KEY_AS_FILENAME -Makes FilesystemIterator::key return the filename.

	      o FilesystemIterator::FOLLOW_SYMLINKS -Makes RecursiveDirectoryIterator::hasChildren follow symlinks.

	      o FilesystemIterator::KEY_MODE_MASK -Masks FilesystemIterator::key

	      o FilesystemIterator::NEW_CURRENT_AND_KEY -Same as FilesystemIterator::KEY_AS_FILENAME | FilesystemIterator::CURRENT_AS_FILEINFO.

	      o FilesystemIterator::SKIP_DOTS -Skips dot files ( . and ..).

	      o FilesystemIterator::UNIX_PATHS -Makes paths use Unix-style forward slash irrespective of system default.

CHANGELOG

       +--------+-------------------------------------------+
       |Version |					    |
       |	|					    |
       |	|		 Description		    |
       |	|					    |
       +--------+-------------------------------------------+
       | 5.3.1	|					    |
       |	|					    |
       |	| Added FilesystemIterator::FOLLOW_SYMLINKS |
       |	|					    |
       +--------+-------------------------------------------+
PHP Documentation Group 												     FILESYSTEMITERATOR(3)