Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

Find all lines in file such that each word on that line appears in at least n lines of the file

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #8  
Old Unix and Linux 06-15-2017
uncleMonty uncleMonty is offline
Registered User
 
Join Date: Jun 2017
Last Activity: 24 June 2017, 9:22 AM EDT
Posts: 5
Thanks: 3
Thanked 0 Times in 0 Posts
Thank you Don. I corrected "cnt[$5]" to "c[5]" and it worked (on bash in OS X 10.6).

---------- Post updated at 02:08 PM ---------- Previous update was at 02:03 PM ----------

Thanks Rudi. That works great. So far I avoided learning awk except for the simplest tasks, because the flow control in snippets I've grabbed here and there is so terse (to me it appears non-existent in your and Don Cragun's solutions). But I've just got hold of Dale Dougherty's book on sed and awk, and I'll be learning how this works now. (Where's the loop for reading the file on the first pass? Where's the conditional print? Etc... I can already see from looking over Dougherty's book that the answers are there, no need to answer.)
Sponsored Links
    #9  
Old Unix and Linux 06-15-2017
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 22 August 2017, 8:42 AM EDT
Location: San Jose, CA, USA
Posts: 10,465
Thanks: 533
Thanked 3,660 Times in 3,123 Posts
Quote:
Originally Posted by uncleMonty View Post
Thank you Don. I corrected "cnt[$5]" to "c[5]" and it worked (on bash in OS X 10.6).

---------- Post updated at 02:08 PM ---------- Previous update was at 02:03 PM ----------

Thanks Rudi. That works great. So far I avoided learning awk except for the simplest tasks, because the flow control in snippets I've grabbed here and there is so terse (to me it appears non-existent in your and Don Cragun's solutions). But I've just got hold of Dale Dougherty's book on sed and awk, and I'll be learning how this works now. (Where's the loop for reading the file on the first pass? Where's the conditional print? Etc... I can already see from looking over Dougherty's book that the answers are there, no need to answer.)
Hi uncleMonty,
I apologize for the typo. It has now been corrected in my earlier post. Note that cnt[$5] should have been changed to c[$5] (NOT c[5])!.

The general form of an awk command (as I'm sure you will find in your book (or the [codei]awk[/icode] man page on your system) is:

Code:
condition { action }

If the condition is not present, the given action is applied to every input line that gets to that statement. If action and the surrounding braces are not present, a default action of print (which prints the current input line after any modifications to that line's contents applied by any previous statements have been applied) is taken for any line in which condition evaluates to a non-zero, non-empty string value. So, the awk statement:

Code:
c[$1] >= cnt && c[$3] >= cnt && c[$5] >= cnt && c[$7] >= cnt

prints any line for which the count of the number of times the contents of fields 1, 3, 5, and 7 have all all been seen cnt or more times.
Sponsored Links
    #10  
Old Unix and Linux 06-16-2017
uncleMonty uncleMonty is offline
Registered User
 
Join Date: Jun 2017
Last Activity: 24 June 2017, 9:22 AM EDT
Posts: 5
Thanks: 3
Thanked 0 Times in 0 Posts
Yes, thanks for the description. And I've learned that the loop through the file the first time to build the array of counters is kept separate from the second loop that prints the line, via the `NR==FNR` trick that I read a good account of in the "two-file processing" section of this webpage: [EDIT: I thought I posted this yesterday but apparently I don't have enough "juice" to give a url on this forum. But the helpful webpage I was just consulting can be found on the backreference.org site with the title "idiomatic awk".]
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
How to find a string in a line in UNIX file and delete that line and previous 3 lines ? vadlamudy UNIX for Advanced & Expert Users 7 05-19-2014 05:25 AM
Read all lines after a string appears in the file. Nagaraja Akkiva Shell Programming and Scripting 4 10-24-2011 07:57 AM
Get last lines of file after last line with word TEST waso Shell Programming and Scripting 6 10-06-2010 08:32 PM
how to find a word in a file that appears next to a given keyword mwrg UNIX for Dummies Questions & Answers 6 09-02-2010 12:06 PM
find uniq lines in file, using the first field of line grom UNIX for Dummies Questions & Answers 8 10-20-2009 05:34 PM



All times are GMT -4. The time now is 11:09 AM.