Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Awk match rows

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Awk match rows
# 1  
Old 01-11-2012
Awk match rows
Hi,

I am pretty new to awk.

I have a text file of the following style

Code:
a b c d e f g h i 1
a b c d e f g h i 2
a b c d e f g h i 3
j k l m n o p q r 4
s t u v w x y z # 5
s t u v w x y z #7

I want the minimum of 10th column if the first 9 columns match with its before and after lines. All columns are separated by tabs.

Desired output:
Code:
a b c d e f g h i 1 
j k l m n o p q r 4
s t u v w x y z # 5

Moderator's Comments:
Mod Comment Video tutorial on how to use code tags in The UNIX and Linux Forums.
Last edited by radoulov; 01-11-2012 at 06:13 PM..
# 2  
Old 01-11-2012
Could you please clarify why the following rows appear in the output?

Code:
j k l m n o p q r 4 
s t u v w x y z # 5

Last edited by radoulov; 01-11-2012 at 06:26 PM..
# 3  
Old 01-11-2012
How about this (output order may be changed):

Code:
awk -F"\t" '
{
  v=$10
  $10=""
  if (!($0 in t)||t[$0] > v) t[$0]=v
}
END { for (k in t) print k t[k]; }' infile

To keep file order the same:
Code:
awk -F"\t" '
{
  v=$10
  $10=""
  if (!($0 in t)||t[$0] > v) {
     t[$0]=v
     N[c++]=$0
  }
}
END {
   for(i=0;i<c;i++)
      print N[i] t[N[i]]  }' infile

# 4  
Old 01-12-2012
Thanks for the code, chubler_xl.

It works fine if there are 2 or 3 similar rows. But, if there are more than 5 or 6 rows that are similar, it doesn't work. It produces more than 2 records for such 5 or 6 similar rows.

Thanks for your help. Further help is appreciated too.

---------- Post updated at 10:47 AM ---------- Previous update was at 10:33 AM ----------

Quote:
Originally Posted by radoulov
Could you please clarify why the following rows appear in the output?

Code:
j k l m n o p q r 4 
s t u v w x y z # 5

That means even though there are no duplicate values for them, it still has to appear because it is an unique record.
# 5  
Old 01-12-2012
Please post a bigger sample of your input data and the expected output based on that input.
# 6  
Old 01-12-2012
Anoter awk script :
Code:
awk '
{
   val = $NF;
   $NF = "";
   key = $0;
   if (key in Mins) {
      if (Mins[key] > val) {
         Mins[key] = val;
      }
   } else {
     Keys[++KeysCnt] = key;
     Mins[key] = val;
   }
}
END {
   for (i=1; i<=KeysCnt; i++) {
      key = Keys[i];
      print key,Mins[key]
   }
}
' inputfile

Inputfile:
Code:
a b c d e f g h i 1
a b c d e f g h i 2
a b c d e f g h i 3
j k l m n o p q r 4
s t u v w x y z # 5
s t u v w x y z # 2
s t u v w x y z # 7

Output:
Code:
a b c d e f g h i  1
j k l m n o p q r  4
s t u v w x y z #  2

Jean-Pierre.
# 7  
Old 01-12-2012
Thanks a lot Jean.

That solved my problem.

Perfect!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk to print match or non-match and select fields/patterns for non-matches

In the awk below I am trying to output those lines that Match between file1 and file2, those Missing in file1, and those missing in file2. Using each $1,$2,$4,$5 value as a key to match on, that is if those 4 fields are found in both files the match, but if those 4 fields are not found then missing... (0 Replies)
Discussion started by: cmccabe
0 Replies

2. Shell Programming and Scripting

awk to update file based on partial match in field1 and exact match in field2

I am trying to create a cronjob that will run on startup that will look at a list.txt file to see if there is a later version of a database using database.txt as the source. The matching lines are written to output. $1 in database.txt will be in list.txt as a partial match. $2 of database.txt... (2 Replies)
Discussion started by: cmccabe
2 Replies

3. Shell Programming and Scripting

awk to match field between two files and use conditions on match

I am trying to look for $2 of file1 (skipping the header) in $2 of file2 (skipping the header) and if they match and the value in $10 is > 30 and $11 is > 49, then print the line from file1 to a output file. If no match is foung the line is not printed. Both the input and output are tab-delimited.... (3 Replies)
Discussion started by: cmccabe
3 Replies

4. Shell Programming and Scripting

Match in awk skipping header rows

I am trying to match $1-$7 between the two files and if a match is found then the contents of $8 in file2 and copied over. The awk I tried is below. There is also a header row in file2 that has the Chr Start End Ref Alt that does not need to be searched. Thank you :). awk awk... (3 Replies)
Discussion started by: cmccabe
3 Replies

5. UNIX for Dummies Questions & Answers

Merge rows into one if first 2 columns match

Hi, I wanted to merge the content and below is input and required output info. Input: /hello,a,r /hello,a,L /hello,a,X /hi,b,v /hi,b,c O/p: /hello,a,r:L:X /hi,v,:v:c Use code tags, thanks. (6 Replies)
Discussion started by: ankitas
6 Replies

6. Shell Programming and Scripting

Rows to Columns with match criteria

Hello Friends, I have a input file having hundreds of rows. I want them to translate in to columns if column 1 is same. Input data: zp06 xxx zp06 rrr zp06 hhh zp06 aaa zp06 ggg zp06 qwer zp06 ser zl11 old3 zl11 old4 zl11 old5 zl11 old6 zl11 old7 zm14 luri zm14 body zm14 ucp (9 Replies)
Discussion started by: suresh3566
9 Replies

7. UNIX for Dummies Questions & Answers

Deleting rows where the value in a specific column match

Hi, I have a tab delimited text file where I want to delete all rows that have the same string for column 1. How do I go about doing that? Thanks! Example Input: aa 1 aa 2 aa 3 bb 4 bc 5 bb 6 cd 8 Output: bc 5 cd 8 (4 Replies)
Discussion started by: evelibertine
4 Replies

8. UNIX for Dummies Questions & Answers

awk display the match and 2 lines after the match is found.

Hello, can someone help me how to find a word and 2 lines after it and then send the output to another file. For example, here is myfile1.txt. I want to search for "Error" and 2 lines below it and send it to myfile2.txt I tried with grep -A but it's not supported on my system. I tried with awk,... (4 Replies)
Discussion started by: eurouno
4 Replies

9. Shell Programming and Scripting

Fetch the rows with match string on a fixed lenth text file - NO delimiters

Hi I am trying to fetch the rows with match string "0000001234" Input file looks like below: 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1 09 0 XXX 0000001234 Z 1... (6 Replies)
Discussion started by: nareshk
6 Replies

10. UNIX for Dummies Questions & Answers

match similar rows. uniq?

hi i have data which is in two columns (such as below). i need to compare two rows against each other and if one row matches the other row (except for different case), and their values in the second column are different, then it prints out one of the rows (either is fine). here is an... (5 Replies)
Discussion started by: Streetrcr
5 Replies
Login or Register to Ask a Question