The UNIX and Linux Forums  

Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting
.
google unix.com



Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
how to read record by record from a file in unix raoscb UNIX for Dummies Questions & Answers 1 05-16-2008 07:30 AM
Script to search a bad record in a file then put the record in the bad file shilendrajadon Shell Programming and Scripting 2 12-28-2007 10:02 AM
Script to search a bad record in a file then put the record in the bad file shilendrajadon UNIX for Advanced & Expert Users 1 12-28-2007 10:00 AM
splitting a record and adding a record to a file rsolap Shell Programming and Scripting 1 08-13-2007 02:58 PM
Select text within matching ( ) bracket cursive UNIX for Dummies Questions & Answers 4 04-20-2007 03:14 AM

Closed Thread
English Japanese Spanish French German Portuguese Italian Dutch Swedish Russian Norwegian Hungarian Hebrew Danish Bulgarian Greek Powered by Powered by Google
 
LinkBack Thread Tools Search this Thread Rate Thread Display Modes
  #1 (permalink)  
Old 06-05-2008
synmag synmag is offline
Registered User
  
 

Join Date: Jun 2008
Posts: 3
select a record from one file matching from second file using awk

I need help

I have two input files and I'd like to generate a report based on the two.

filea:
hostname,account1,password
,account2,password
hostname,account1,password
hostname,account1,password
,account1,password
,account2,password

repeating hostnames are blank

fileb:
hosta
hostb
hostc

I need to generate a record of "hostname account password" for all hosts in fileb that occur in filea for a specific account. I.e. I want a list of root passwords for all hosts in fileb. I know the logic I want to use but don't have the scripting skills to implement it. I think the easiest way would be to use awk to fill in the blanks with the hostname of filea and then simply use a shell script to grep for each host in fileb of the new filea.

The other way is to run down filea for each hostname in fileb until either the account is matched or the hostname is different and print the result. If the account does not exist for the given hostname do nothing.

I'm trying to save manually doing this by scripting it thinking it would be faster but maybe I was wrong

Here is the framework I came up with so far for the harder way because it would be more elegant

#!/bin/sh
#
hosts=`cat hosts.txt`
hostname="x"
last="x"

for name in ${hosts}
do
#the awk logic would come here
done
  #2 (permalink)  
Old 06-05-2008
lo tan lo tan is offline
Registered User
  
 

Join Date: May 2008
Posts: 8
you may give us an example of the result that you try to achieve and base on that we may come up with a solution.
  #3 (permalink)  
Old 06-06-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
  
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,652
You mean something like this?


Code:
sed -e 's/.*/^&,/' fileb | grep -f - filea

Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
  #4 (permalink)  
Old 06-06-2008
synmag synmag is offline
Registered User
  
 

Join Date: Jun 2008
Posts: 3
Quote:
Originally Posted by era View Post
You mean something like this?


Code:
sed -e 's/.*/^&,/' fileb | grep -f - filea

Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
Thanks for the reply. My version does not support -f unfortunately. Looking at the code where would I specify the account name I'm looking for? Say if I want to list the host,account,password for all root accounts? Also would this work for records where the host name is blank?

To clarify the problem, filea has blanks for repeating hostnames and if say root is in a record where there is no hostname how do you derive the hostname for the record? The hostname I'm looking for is the prior non blank record.

example:

hosta,johndoe,abc123
,root,4009dlkj
hostb,janedoe,rrrrrr

if host a is in fileb the expected out put for root is

hosta,root,4009dlkj

I was trying to use awk to replace the blank with a hostname from the previous nonblank record but I kept getting errors. I couldn't even store $1 to a variable and print it as a test. I know sed is very powerful but I didn't think this could be done with one line.

I ended up doing it manually last night. At this point it's inquiring minds want to know and it might come in handy if I have to do it again.

TIA
  #5 (permalink)  
Old 06-06-2008
reborg's Avatar
reborg reborg is online now Forum Staff  
Administrator
  
 

Join Date: Mar 2005
Location: Ireland
Posts: 4,245
Quote:
Originally Posted by era View Post
Not all grep versions support the -f option apparently, but the idea is simply to grep for anything from fileb which is in the first field of filea.
True, but most that don't will support either egrep -f or fgrep.
  #6 (permalink)  
Old 06-09-2008
era era is offline Forum Advisor  
Herder of Useless Cats (On Sabbatical)
  
 

Join Date: Mar 2008
Location: /there/is/only/bin/sh
Posts: 3,652
Oh, I misunderstood your problem description. Here's something which hopefully is closer to what you wanted.


Code:
awk -F , -v acct=root 'NR==FNR { h[$1]++; next; }
{ if($1 == "") $1=host; host=$1; if (! h[$1]) next;
if ($2 == acct) print }' fileb filea

The NR==FNR condition is true while fileb is being read. This is a common awk idiom for reading in a file of auxiliary data before the main processing. The host names in fileb will be used to populate the array h. Then in the main body of the script, if the first field is empty, the value of the host variable will be used for $1. Then the current value of $1 will be remembered in host in case the next line(s) have empty first fields. Then, if the array h does not contain the current host name (meaning it was not present in fileb), the current line is skipped. Finally, if the second field is identical to the variable acct, the line is printed.

(Really old variants of awk do not support passing in variables with the -v option -- if you have this problem, see if you can find nawk or mawk or gawk on your system, or an XPG4 awk. Or you can interpolate the variable directly into the script.)
  #7 (permalink)  
Old 06-11-2008
synmag synmag is offline
Registered User
  
 

Join Date: Jun 2008
Posts: 3
Quote:
Originally Posted by era View Post
Oh, I misunderstood your problem description. Here's something which hopefully is closer to what you wanted.


Code:
awk -F , -v acct=root 'NR==FNR { h[$1]++; next; }
{ if($1 == "") $1=host; host=$1; if (! h[$1]) next;
if ($2 == acct) print }' fileb filea

The NR==FNR condition is true while fileb is being read. This is a common awk idiom for reading in a file of auxiliary data before the main processing. The host names in fileb will be used to populate the array h. Then in the main body of the script, if the first field is empty, the value of the host variable will be used for $1. Then the current value of $1 will be remembered in host in case the next line(s) have empty first fields. Then, if the array h does not contain the current host name (meaning it was not present in fileb), the current line is skipped. Finally, if the second field is identical to the variable acct, the line is printed.

(Really old variants of awk do not support passing in variables with the -v option -- if you have this problem, see if you can find nawk or mawk or gawk on your system, or an XPG4 awk. Or you can interpolate the variable directly into the script.)
That's exactly what I wanted. My version of awk didn't support the passing of the variable but I do have nawk and it worked perfectly.

Based on awk documentation I was trying to use the begin block to read the file but couldn't figure out how to read from two files. Would you mind explaining how awk determines which file to read from? Is it the code in the first set of braces {} reads the first file supplied and the second set the second file?

Thank you. I learned a lot from this!
Closed Thread

Bookmarks

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes Rate This Thread
Rate This Thread:

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On




All times are GMT -4. The time now is 04:17 PM.


Powered by: vBulletin, Copyright ©2000 - 2006, Jelsoft Enterprises Limited. Language Translations Powered by .
vBCredits v1.4 Copyright ©2007 - 2008, PixelFX Studios
The UNIX and Linux Forums Content Copyright ©1993-2009. All Rights Reserved.Ad Management by RedTyger

Content Relevant URLs by vBSEO 3.2.0