I am trying to use awk to remove all rows where the first 4 fields are duplicates. e.g. in the following data lines 6-9 would be removed, leaving one copy of the duplicated row (row 5)
so the output would hopefully look like
Can anyone help? Thanks
Last edited by radoulov; 10-28-2011 at 06:45 AM..
Reason: Code tags!
i have the long file more than one ns and www and mx in the line like .
i need the first ns record and first www and first mx from line .
the records are seperated with tthe ; i am try ing in awk scripting not getiing the solution.
... (4 Replies)
Hello,
I met a challenge to extract part of the table. I'd like to grep the first three matches based on field1 and field2. Input:
D A 92.85 1315 83 11
D A 95.90 757 28 3
D A 94.38 480 20 7
D A 91.21 307 21 6
D A 94.26 244 ... (6 Replies)
Hello,
I am trying to eliminate rows where the first field is duplicated, leaving the row where the last field is "NET".
Data file:
345234|22.34|LST
546543|55.33|LST
793929|98.23|LST
793929|64.69|NET
149593|49.22|LST
Desired output:
345234|22.34|LST
546543|55.33|LST... (2 Replies)
In a folder I'll several times daily receive new files that I want to combine into one big file, without any duplicate rows.
The file name in the folder will look like e.q:
MissingData_2014-08-25_09-30-18.txt
MissingData_2014-08-25_09-30-14.txt
MissingData_2014-08-26_09-30-12.txt
The content... (9 Replies)
Hello everyone, I have a dataset that looks something like:
1 3
2 2
3 4,5
4 3:9
5 5,9
6 5:6
I need to remove the rows that contain a comma in the second column and I'm not sure how to go about this. Here is an attempt.
awk 'BEGIN {FS=" "} { if ($2!==,) print }'Any help is appreciated. (5 Replies)
I am trying to cut a range of fields in awk. The below seems to work for removing field 50, but what is the correct syntax for removing a range ($50-$62). Thank you :).
awk
awk 'BEGIN{FS=OFS="\t"}{$50=""; gsub(/\t\t/,"\t")}1' test.vcf.hg19_multianno.txt > output.csv
Maybe:
awk... (6 Replies)
I am trying to remove all the lines and spaces where the count in $4 or $5 is greater than 1 (more than 1 letter). The file and the output are tab-delimited. Thank you :).
file
X 5811530 . G C NLGN4X
17 10544696 . GA G MYH3
9 96439004 . C ... (1 Reply)
Discussion started by: cmccabe
1 Replies
LEARN ABOUT REDHAT
igawk
IGAWK(1) Utility Commands IGAWK(1)NAME
igawk - gawk with include files
SYNOPSIS
igawk [ all gawk options ] -f program-file [ -- ] file ...
igawk [ all gawk options ] [ -- ] program-text file ...
DESCRIPTION
Igawk is a simple shell script that adds the ability to have ``include files'' to gawk(1).
AWK programs for igawk are the same as for gawk, except that, in addition, you may have lines like
@include getopt.awk
in your program to include the file getopt.awk from either the current directory or one of the other directories in the search path.
OPTIONS
See gawk(1) for a full description of the AWK language and the options that gawk supports.
EXAMPLES
cat << EOF > test.awk
@include getopt.awk
BEGIN {
while (getopt(ARGC, ARGV, "am:q") != -1)
...
}
EOF
igawk -f test.awk
SEE ALSO gawk(1)
Effective AWK Programming, Edition 1.0, published by the Free Software Foundation, 1995.
AUTHOR
Arnold Robbins (arnold@skeeve.com).
Free Software Foundation Nov 3 1999 IGAWK(1)