Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

reading in a file through

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting reading in a file through
# 8  
Old 05-16-2012
Quote:
Originally Posted by agama
Small change should ignore the secondary address:
A simple change to the regex would also do the trick (replace "<spc>" and "<tab>" with literal tabs and spaces):

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s\n", name, ip );
        name = "";
    }
    /^[<spc><tab>]*host_name[<spc><tab>]/ { name = $2; next; }
    /^[<spc><tab>]*address[<spc><tab>]/ { ip = $2; next; }
' host.conf chunk-file >output-file

How the script works: first the list of host names to search for is read into the array "seen". Then the file with the chunks is scanned.

The script reacts to three different type of lines in the chunk file:

1st type: line starts with "}". This is a line ending a chunk and if the variable "name" contains a hostname corresponding to one in the seen-array then the info (variables "name" and "ip") gathered before is printed out and "name" is cleared again.

2nd type: line starts with some optional spaces/tabs followed by the word "host_name" followed by a space or tab. The second part of this line is stored in the variable "name" for use when the next type-1-line is found. Then the rest of the script is skipped for this line and the next line of the chunk file is read.

3rd type: line starts with some optional spaces/tabs followed by the word "address" followed by a space or tab. The procedure is the same as with type-2-lines, but this time the second part of the line is stored in the variable "ip".

Notice that by surrounding the keywords with white space the ambiguity between "address" and "secondary_address" is automatically resolved.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
agama
# 9  
Old 05-16-2012
Capturing both IP addresses:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s ---- %s\n", name, ip, ip2 );   # now prints both
        name = "";
        snarf = 0;             # end of chunk turn capture off
    }
    /_secondary_address/ { ip2 = $2; next}       # captures secondary too
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }

This User Gave Thanks to agama For This Post:
SkySmart
# 10  
Old 05-16-2012
Quote:
Originally Posted by agama
Capturing both IP addresses:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s ---- %s\n", name, ip, ip2 );   # now prints both
        name = "";
        snarf = 0;             # end of chunk turn capture off
    }
    /_secondary_address/ { ip2 = $2; next}       # captures secondary too
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }

unfortunately, this grabs IPs that belong to other server's chunks. there are some chunks for some servers that do not have a "_secondary_address" ip assigned. they just have ip for "address" assigned. for those, i expected the 3rd field to be empty since theres nothing to grab.

here's a script i was using but for some reason, this script doesn't read the entire host list i gave it, even though all the conditions are met. for instance, say there are 100 hosts in a list. it will for some reason not scan through all 100. it omits servers. i dont know which part of the script is making it do so.

Code:
awk 'BEGIN {
  while((getline < "host.list")>0)
     S[$0]

  FS="\n"; RS="}\n"; ORS="}\n";
}

/define host/ {

  for(X in D) delete D[X];

  for(N=2; N<=NF; N++)
  {
       split($N, A, " ");
       D[A[1]]=A[2];
  }

  if (D["host_name"] in S)
       printf("%s -------- %s -------- %s\n", D["host_name"]" " " ", D["address"], D["_secondary_address"])

}' host.conf

hosts.conf is the file that contains the "chunks".
# 11  
Old 05-16-2012
Silly assumption on my part. Setting both address variables to empty strings after printing should be all that is needed:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s ---- %s\n", name, ip, ip2 );   # now prints both
        name = = ip, ip2 = ""    # reset everything; 
        snarf = 0;             # end of chunk turn capture off
    }
    /_secondary_address/ { ip2 = $2; next}       # captures secondary too
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
'  host.list host.conf

Last edited by agama; 05-16-2012 at 08:43 PM.. Reason: typo
This User Gave Thanks to agama For This Post:
SkySmart
# 12  
Old 05-16-2012
Quote:
Originally Posted by agama
Silly assumption on my part. Setting both address variables to empty strings after printing should be all that is needed:

Code:
awk '
    NR == FNR { seen[$1]; next }   # collect hosts from the conf file
    /define host.*{/ { snarf = 1; next; }    # ok to capture data after this point
    !snarf { next; }              # skip record if not capturing
    /}/ {                        # end of chunk, if name was in conf list, print data
        if( name in seen )
            printf( "%20s ---- %s ---- %s\n", name, ip, ip2 );   # now prints both
        name = = ip, ip2 = ""    # reset everything; 
        snarf = 0;             # end of chunk turn capture off
    }
    /_secondary_address/ { ip2 = $2; next}       # captures secondary too
    /host_name/ { name = $2; next; }
    /address/ { ip = $2; next; }
'  host.list host.conf

I ran this code, i got an error:

Code:
awk: cmd. line:8:         name = = ip, ip2 = ""    # reset everything; 
awk: cmd. line:8:                ^ syntax error

# 13  
Old 05-16-2012
Quote:
Originally Posted by SkySmart
here's a script i was using but for some reason, this script doesn't read the entire host list i gave it, even though all the conditions are met. for instance, say there are 100 hosts in a list. it will for some reason not scan through all 100. it omits servers. i dont know which part of the script is making it do so.

Code:
awk 'BEGIN {
  while((getline < "host.list")>0)
     S[$0]

  FS="\n"; RS="}\n"; ORS="}\n";
}

/define host/ {

  for(X in D) delete D[X];

  for(N=2; N<=NF; N++)
  {
       split($N, A, " ");
       D[A[1]]=A[2];
  }

  if (D["host_name"] in S)
       printf("%s -------- %s -------- %s\n", D["host_name"]" " " ", D["address"], D["_secondary_address"])

}' host.conf

hosts.conf is the file that contains the "chunks".
I cut/pasted your code and for my small set of data it does just fine. I gave it 100 dummy names in addition to the ones you had in your example. Can you post a list of what seems not being captured from your list?
This User Gave Thanks to agama For This Post:
SkySmart
# 14  
Old 05-16-2012
Quote:
Originally Posted by agama
I cut/pasted your code and for my small set of data it does just fine. I gave it 100 dummy names in addition to the ones you had in your example. Can you post a list of what seems not being captured from your list?

i guess it would help to point out that the chunks are all entered into the file by hand, and sometimes by an automated script.

so for some of the chunks, there are tabs in there, or spaces, or both.

didn't think that would matter.

i wish i can post more information about the files in question but i cant due to security reasons.

however, i would prefer to use your latest updated script instead. the one i said i got an error on. i think what may be missing in that script may just be something very simple. can you please look it over?
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

ksh Script, Reading A File, Grepping A File Contents In Another File

So I'm stumped. First... APOLOGIES... my work is offline in an office that has zero internet connectivity, as required by our client. If need be, I could print out my script attempts and retype them here. But on the off chance... here goes. I have a text file (file_source) of terms, each line... (3 Replies)
Discussion started by: Brusimm
3 Replies

2. UNIX for Dummies Questions & Answers

Reading Xml file and print the values into the text file in columnwise?

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (4 Replies)
Discussion started by: sravanreddy
4 Replies

3. UNIX for Dummies Questions & Answers

Reading XML file and print the values in the text file using Linux shell script

hi guys, i want help... Reding XML file and print the values into the text file using linux shell script file as per below xml file <sequence> <Filename>aldorzum.doc</Filename> <DivisionCode>US</DivisionCode> <ContentType>Template</ContentType> <ProductCode>VIMZIM</ProductCode> </sequence>... (1 Reply)
Discussion started by: sravanreddy
1 Replies

4. Shell Programming and Scripting

Reading UNIX commands from file and redirecting output to a file

Hi All I have written the following script: #!/bin/ksh while read cmdline do echo `$cmdline` pid="$cmdline" done<commands.txt =========== commands.txt contains: ps -ef | grep abc | grep xyz |awk '{print $2}; My objective is to store the o/p of the command in a variable and do... (8 Replies)
Discussion started by: rahulparo
8 Replies

5. Shell Programming and Scripting

fatal: cannot open file `TNAME' for reading (No such file or directory)

Hi, I am running this command through a shell script and getting the error mentioned in the subject line: testing.awk -f x.txt TNAME My testing.awk file contains something like ++++++++++++++++++ #!/usr/bin/awk -f BEGIN{ TAB_NAME="INSERT_ONE_" ARGV ; } if ( $1=="JAM_ONE" &&... (1 Reply)
Discussion started by: kunwar
1 Replies

6. Shell Programming and Scripting

Searching for Log / Bad file and Reading and writing to a flat file

Need to develop a unix shell script for the below requirement and I need your assistance: 1) search for file.log and file.bad file in a directory and read them 2) pull out "Load_Start_Time", "Data_File_Name", "Error_Type" from log file 4) concatinate each row from bad file as... (3 Replies)
Discussion started by: mlpathir
3 Replies

7. UNIX for Dummies Questions & Answers

Reading from a file(passing the file as input parameter)

hi I have a shell script say primary.sh . There is a file called params my scenario is primary.sh should read all the values and echo it for example i should pass like $primary.sh params output would be Abc ... (2 Replies)
Discussion started by: ssuresh1999
2 Replies

8. UNIX for Advanced & Expert Users

Reading a file and sending mail based on content of the file

Hi Gurus, I am having an requirement. i have to read a list file which contains file names and send mail to different users based on the files in the list file. eg. if file a.txt exists then send a mail to a@a.com simillary for b.txt,c.txt etc. Thanks for your help, Nimu (6 Replies)
Discussion started by: nimu1979
6 Replies

9. Shell Programming and Scripting

Reading file names from a file and executing the relative file from shell script

Hi How can i dynamically read files names from a list file and execute them from a single shell script. Please help its urgent Thanks in Advance (4 Replies)
Discussion started by: anushilrai
4 Replies
Login or Register to Ask a Question