The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
Google UNIX.COM


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
how to read record by record from a file in unix raoscb UNIX for Dummies Questions & Answers 1 05-16-2008 03:30 AM
Script to search a bad record in a file then put the record in the bad file shilendrajadon Shell Programming and Scripting 2 12-28-2007 06:02 AM
Script to search a bad record in a file then put the record in the bad file shilendrajadon UNIX for Advanced & Expert Users 1 12-28-2007 06:00 AM
splitting a record and adding a record to a file rsolap Shell Programming and Scripting 1 08-13-2007 10:58 AM
Select text within matching ( ) bracket cursive UNIX for Dummies Questions & Answers 4 04-19-2007 11:14 PM

Reply
 
LinkBack Thread Tools Display Modes
  #1 (permalink)  
Old 06-05-2008
Registered User
 

Join Date: Jun 2008
Posts: 3
select a record from one file matching from second file using awk

I need help

I have two input files and I'd like to generate a report based on the two.

filea:
hostname,account1,password
,account2,password
hostname,account1,password
hostname,account1,password
,account1,password
,account2,password

repeating hostnames are blank

fileb:
hosta
hostb
hostc

I need to generate a record of "hostname account password" for all hosts in fileb that occur in filea for a specific account. I.e. I want a list of root passwords for all hosts in fileb. I know the logic I want to use but don't have the scripting skills to implement it. I think the easiest way would be to use awk to fill in the blanks with the hostname of filea and then simply use a shell script to grep for each host in fileb of the new filea.

The other way is to run down filea for each hostname in fileb until either the account is matched or the hostname is different and print the result. If the account does not exist for the given hostname do nothing.

I'm trying to save manually doing this by scripting it thinking it would be faster but maybe I was wrong

Here is the framework I came up with so far for the harder way because it would be more elegant

#!/bin/sh
#
hosts=`cat hosts.txt`
hostname="x"
last="x"

for name in ${hosts}
do
#the awk logic would come here
done
Reply With Quote
Forum Sponsor
  #2 (permalink)  
Old 06-05-2008
Registered User
 

Join Date: May 2008
Posts: 7
you may give us an example of the result that you try to achieve and base on that we may come up with a solution.
Reply With Quote
  #3 (permalink)  
Old 06-05-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 2,923
You mean something like this?

Code:
sed -e 's/.*/^&,/' fileb | grep -f - filea
Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
Reply With Quote
  #4 (permalink)  
Old 06-06-2008
Registered User
 

Join Date: Jun 2008
Posts: 3
Quote:
Originally Posted by era View Post
You mean something like this?

Code:
sed -e 's/.*/^&,/' fileb | grep -f - filea
Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
Thanks for the reply. My version does not support -f unfortunately. Looking at the code where would I specify the account name I'm looking for? Say if I want to list the host,account,password for all root accounts? Also would this work for records where the host name is blank?

To clarify the problem, filea has blanks for repeating hostnames and if say root is in a record where there is no hostname how do you derive the hostname for the record? The hostname I'm looking for is the prior non blank record.

example:

hosta,johndoe,abc123
,root,4009dlkj
hostb,janedoe,rrrrrr

if host a is in fileb the expected out put for root is

hosta,root,4009dlkj

I was trying to use awk to replace the blank with a hostname from the previous nonblank record but I kept getting errors. I couldn't even store $1 to a variable and print it as a test. I know sed is very powerful but I didn't think this could be done with one line.

I ended up doing it manually last night. At this point it's inquiring minds want to know and it might come in handy if I have to do it again.

TIA
Reply With Quote
  #5 (permalink)  
Old 06-06-2008
reborg's Avatar
Administrator
 
Join Date: Mar 2005
Location: Ireland
Posts: 3,498
Quote:
Originally Posted by era View Post
Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
True, but most that don't will support either egrep -f or fgrep.
Reply With Quote
  #6 (permalink)  
Old 06-09-2008
era era is offline
Herder of Useless Cats
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 2,923
Oh, I misunderstood your problem description. Here's something which hopefully is closer to what you wanted.

Code:
awk -F , -v acct=root 'NR==FNR { h[$1]++; next; }
{ if($1 == "") $1=host; host=$1; if (! h[$1]) next;
if ($2 == acct) print }' fileb filea
The NR==FNR condition is true while fileb is being read. This is a common awk idiom for reading in a file of auxiliary data before the main processing. The host names in fileb will be used to populate the array h. Then in the main body of the script, if the first field is empty, the value of the host variable will be used for $1. Then the current value of $1 will be remembered in host in case the next line(s) have empty first fields. Then, if the array h does not contain the current host name (meaning it was not present in fileb), the current line is skipped. Finally, if the second field is identical to the variable acct, the line is printed.

(Really old variants of awk do not support passing in variables with the -v option -- if you have this problem, see if you can find nawk or mawk or gawk on your system, or an XPG4 awk. Or you can interpolate the variable directly into the script.)
Reply With Quote
  #7 (permalink)  
Old 06-11-2008
Registered User
 

Join Date: Jun 2008
Posts: 3
Quote:
Originally Posted by era View Post
Oh, I misunderstood your problem description. Here's something which hopefully is closer to what you wanted.

Code:
awk -F , -v acct=root 'NR==FNR { h[$1]++; next; }
{ if($1 == "") $1=host; host=$1; if (! h[$1]) next;
if ($2 == acct) print }' fileb filea
The NR==FNR condition is true while fileb is being read. This is a common awk idiom for reading in a file of auxiliary data before the main processing. The host names in fileb will be used to populate the array h. Then in the main body of the script, if the first field is empty, the value of the host variable will be used for $1. Then the current value of $1 will be remembered in host in case the next line(s) have empty first fields. Then, if the array h does not contain the current host name (meaning it was not present in fileb), the current line is skipped. Finally, if the second field is identical to the variable acct, the line is printed.

(Really old variants of awk do not support passing in variables with the -v option -- if you have this problem, see if you can find nawk or mawk or gawk on your system, or an XPG4 awk. Or you can interpolate the variable directly into the script.)
That's exactly what I wanted. My version of awk didn't support the passing of the variable but I do have nawk and it worked perfectly.

Based on awk documentation I was trying to use the begin block to read the file but couldn't figure out how to read from two files. Would you mind explaining how awk determines which file to read from? Is it the code in the first set of braces {} reads the first file supplied and the second set the second file?

Thank you. I learned a lot from this!
Reply With Quote
Google UNIX.COM
Reply

Thread Tools
Display Modes




All times are GMT -7. The time now is 12:37 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited.
The UNIX and Linux Forums Content Copyright ©1993-2008 The CEP Blog All Rights Reserved -Ad Management by RedTyger Visit The Global Fact Book

Content Relevant URLs by vBSEO 3.2.0