reading in a file through


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting reading in a file through
# 1  
Old 05-15-2012
reading in a file through

I'm trying to use the below script to perform a certain task.

task is: i have a file that has data that is similar to what is pasted below.

we'll call this file host.conf

Code:
define host {
      address 10.11.11.14
      alias mail-server
      host_name        mail-server01.sky.net
      use generic-host
}

What I just pasted above is what i refer to as a chunk. Now, there are several of these chunks in the host.conf file.

now, i have a separate file that contains a list of host names. we'll call this separate file host.list.

the list of hostnames in host.list is one server per line.
Code:
example:

mail-server01.sky.net
mail-server02.sky.net
mail-server03.sky.net

i want to take each host that is in host.list, and run it against the host.conf file. what i'm looking to do is to get the IP of the host from the host.conf file.

in other words, every host that I have in host.list has a configuration set up in host.conf. the configuration set up for each host is the chunk i pasted above.

below is the code im attempting to use to help me pull out what i want.

Code:
awk 'BEGIN {
  while((getline < "host.list")>0)
     S[$0]

  FS="\n"; RS="}\n"
}

/define host/ && /}/ {

  for(X in D) delete D[X];

  for(N=2; N<=NF; N++)
  {
       split($N, A, " ");
       D[A[1]] = A[2]
       i = 3;
       while (i in A) 
          D[A[1]] = D[A[1]] "=" A[i++];
  }

  if (D["host_name"] in S) 
       printf("%20s -- %50s\n", D["host_name"], D["address"])

}' $1

the output of this script should be:

Code:
hostname ---- ip
ex:
mail-server01.sky.net  ------  10.11.11.14

my os is: redhat and sunos
shell: bash

Last edited by SkySmart; 05-15-2012 at 08:37 PM..
# 2  
Old 05-15-2012
I'm no guru, and I'm sure there's a one-liner way to do it with awk or something, but if pressed for time, here's how I would do it in a shell script:

Code:
 
#!/bin/bash
 
#first, put each chunk from the host.conf file on a single line of a temp file
#and make sure all delimiters are single spaces, and there are none leading
echo -ne `cat host.conf` | sed 's/}/}\n/g' | sed 's/^ *//;s/  */ /g' > host.conf.searchable
 
#then read each line from the host.list file one at a time
while read HOSTNAME
do
  #check the temp file to see if there is a matching entry in the host.conf file
  if [[ -n grep $HOSTNAME host.conf.searchable ]] ; then
    #if there is, get the IP field of the matching line
    IP=`grep $HOSTNAME host.conf.searchable |cut -d" " -f4`
    #and then just spit it out in the required format
    echo "$HOSTNAME ----- $IP"
  fi
done<host.list
 
#my mama always told me to clean up after myself
rm host.conf.searchable
 
exit


Last edited by Finja; 05-15-2012 at 10:36 PM..
This User Gave Thanks to Finja For This Post:
# 3  
Old 05-15-2012
I think the awk solution can be a wee bit more simple:

Code:
   awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
    }
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



Doesn't assume that elements in the block have a specific order; this is an advantage if chunk file is hand coded.
This User Gave Thanks to agama For This Post:
# 4  
Old 05-16-2012
Code:
grep -f host.list <(sed -e '/address\|host_name/!d' host.conf  | sed -n 'h;n;H;x;s/\n/ /;p') | awk '{print $NF,"------",$2}'

This User Gave Thanks to complex.invoke For This Post:
# 5  
Old 05-16-2012
Quote:
Originally Posted by agama
I think the awk solution can be a wee bit more simple:

Code:
   awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
    }
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



Doesn't assume that elements in the block have a specific order; this is an advantage if chunk file is hand coded.
thank you. this is the code i need. it works.

one question, where do you specify the "define host"? how does the script know where to begin?

i ask because in each chunks, there can be two ips set for one host. the ips will be specified by:

Code:
address  10.11.11.45
secondary_address 10.19.19.19

looks like the script only grabs any field that matches "address", when in fact, i need it to grab only the primary address, which is the "10.11.11.45".

---------- Post updated at 01:35 PM ---------- Previous update was at 09:54 AM ----------

It seems the script is grabbing the wrong IPs.

i modified the script to, instead of grabbing the primary ip that is defined in a chunk like this:

Code:
 define host {
      address 10.11.11.14
      alias mail-server
      host_name        mail-server01.sky.net
      use generic-host
}

i modified the script to grab the secondary IP instead. but its not working. it's grabbing ips that dont belong to the chunk of the particular host it is saying it belongs to.

Code:
 define host {
      alias mail-server
      host_name        mail-server01.sky.net
      _secondary_address 10.11.11.14
      use generic-host
}

here's my script, i'm sure its wrong:

Code:
  awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "F";
    }
    /host_name/ { name = $2; next; }
    / _secondary_address/ { ip = $2; next; }

' host.list host.conf

need help please
# 6  
Old 05-16-2012
Small change should ignore the secondary address:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
    }
    /secondary_address/ { next; }
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



And to answer your original question, the programme doesn't look for define host tags. Given what you posted as the contents of your file it doesn't need to. If there are other 'chunks' that need to be ignored, then it might not do the right thing depending on the contents of thoese chunks.

something like this would help if there are multiple chunks to be ignored:
Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
        snarf = 0;             # end of chunk turn capture off
    }
    /secondary_address/ { next; }     # ignore secondary address edit: this might need to be /_secondary_address/
    /host_name/ { name = $2; next; }  
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



I didn't test this, so it could have a typo, but it looks clean to me.

---------- Post updated at 18:10 ---------- Previous update was at 18:02 ----------

I just reread your last posts and am a bit confused. Is the secondary address marked with secondary_address or does the string have a leading underbar? Given that your programme is looking to capture with the leading underbar, and it's not working, I'm assuming it doesn't have it in the file. The problem might also have been leading whitespace. If the configuration file records don't have any whitespace, or it's tabs and not at least a space, the leading space in your pattern (/ _secondary_address/ could be your problem too. It's hard to tell since you only posted bits and pieces of your config file.

I hope this makes some sense, and that some of it gets you going again.

Last edited by agama; 05-16-2012 at 07:11 PM.. Reason: added comment
This User Gave Thanks to agama For This Post:
# 7  
Old 05-16-2012
Quote:
Originally Posted by agama
Small change should ignore the secondary address:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
    }
    /secondary_address/ { next; }
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



And to answer your original question, the programme doesn't look for define host tags. Given what you posted as the contents of your file it doesn't need to. If there are other 'chunks' that need to be ignored, then it might not do the right thing depending on the contents of thoese chunks.

something like this would help if there are multiple chunks to be ignored:
Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
        snarf = 0;             # end of chunk turn capture off
    }
    /secondary_address/ { next; }     # ignore secondary address edit: this might need to be /_secondary_address/
    /host_name/ { name = $2; next; }  
    /address/ { ip = $2; next; }
' host.conf chunk-file >output-file



I didn't test this, so it could have a typo, but it looks clean to me.

---------- Post updated at 18:10 ---------- Previous update was at 18:02 ----------

I just reread your last posts and am a bit confused. Is the secondary address marked with secondary_address or does the string have a leading underbar? Given that your programme is looking to capture with the leading underbar, and it's not working, I'm assuming it doesn't have it in the file. The problem might also have been leading whitespace. If the configuration file records don't have any whitespace, or it's tabs and not at least a space, the leading space in your pattern (/ _secondary_address/ could be your problem too. It's hard to tell since you only posted bits and pieces of your config file.

I hope this makes some sense, and that some of it gets you going again.
thank you so much.

this works:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
        snarf = 0;             # end of chunk turn capture off
    }
    /_secondary_address/ { next; }     # ignore secondary address edit: this might need to be /_secondary_address/
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }

i just need one last thing. i need to modify this code to not spit out both the ips for address and _secondary address. so i dont want to ignore secondary address anymore. i want this code to spit out something like this:

Code:
sky.net ---- 10.11.11.11 ---- 10.12.12.12

in this case of course, the first ip will be the ip provided for "address", and the second ip will be the one provided for "_secondary_address".

thanks again!
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. UNIX for Dummies Questions & Answers

Reading Xml file and print the values into the text file in columnwise?

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (4 Replies)
Discussion started by: sravanreddy
4 Replies

3. UNIX for Dummies Questions & Answers

Reading XML file and print the values in the text file using Linux shell script

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (1 Reply)
Discussion started by: sravanreddy
1 Replies

4. Shell Programming and Scripting

Reading UNIX commands from file and redirecting output to a file

Hi All I have written the following script: #!/bin/ksh while read cmdline do echo `$cmdline` pid="$cmdline" done<commands.txt =========== commands.txt contains: ps -ef | grep abc | grep xyz |awk '{print $2}; My objective is to store the o/p of the command in a variable and do... (8 Replies)
Discussion started by: rahulparo
8 Replies

5. Shell Programming and Scripting

fatal: cannot open file `TNAME' for reading (No such file or directory)

Hi, I am running this command through a shell script and getting the error mentioned in the subject line: testing.awk -f x.txt TNAME My testing.awk file contains something like ++++++++++++++++++ #!/usr/bin/awk -f BEGIN{ TAB_NAME="INSERT_ONE_" ARGV ; } if ( $1=="JAM_ONE" &&... (1 Reply)
Discussion started by: kunwar
1 Replies

6. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

7. UNIX for Dummies Questions & Answers

Reading from a file(passing the file as input parameter)

hi I have a shell script say primary.sh . There is a file called params my scenario is primary.sh should read all the values and echo it for example i should pass like $primary.sh params output would be Abc ... (2 Replies)
Discussion started by: ssuresh1999
2 Replies

8. UNIX for Advanced & Expert Users

Reading a file and sending mail based on content of the file

Hi Gurus, I am having an requirement. i have to read a list file which contains file names and send mail to different users based on the files in the list file. eg. if file a.txt exists then send a mail to a@a.com simillary for b.txt,c.txt etc. Thanks for your help, Nimu (6 Replies)
Discussion started by: nimu1979
6 Replies

9. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question