Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

awk - remove block of text, multiple actions for 'if', inline edit

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users awk - remove block of text, multiple actions for 'if', inline edit
# 1  
Old 10-26-2010
awk - remove block of text, multiple actions for 'if', inline edit
I'm having a couple of issues. I'm trying to edit a nagios config and remove a host definition if a certain "host_name" is found. My thought is I would find host definition block containing the host_name I'm looking for and output the line numbers for the first and last lines. Using set, I will assign those as variables in my bash script and use another awk to output all lines before the start of the block and all the lines after the end of the block to a new file and replace the current conf with the output file.

A snippet of the Nagios conf file looks like this:
Code:
define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-1-1-1
    alias        XMPP_ec2-184-1-1-1
    address        ip-10-10-10-10.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-2-2-2
    alias        XMPP_ec2-184-2-2-2
    address        ip-10-20-20-20.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-204-1-1-1
    alias        XMPP_ec2-204-1-1-1
    address        ip-10-30-30-30.us-west-1.compute.internal
    }

So, I want to remove the block containing "host_name" of "XMPP_ec2-184-2-2-2" and the blank line after it. I've come up with this one-liner which will output the line number of "define host{" and the line number of the line after the closing curly brace for that block:
Code:
awk -v hostname="XMPP_ec2-184-2-2-2" '/define host/ {startblock=NR}; {if ($1 == "host_name" && $2 ~ hostname) foundblock=1}; \
{if (/^\t\}/ && foundblock == 1) printf "%s %s\n", startblock, NR+1} {if (/^\t\}/ && foundblock == 1) foundblock=0}' nagiosconfigfile.cfg

which outputs:
Code:
8 14

This is the correct output. My questions are:

  1. I have to put in "if (/^\t\}/ && foundblock == 1)" twice because I want it to output the line numbers and also set "foundblock=0". How can I perform multiple actions if the condition is true?
  2. or, since I'm confident the host will only be defined once, how do I output and stop parsing the file so I don't have to reset "foundblock" to 0?
  3. I plan on using "set --" to output that 8 and 14 to 2 variables within a bash script. Then I will go back through the file and output all lines but those with:
    Code:
    awk -v startrow=$start -v endrow=$end '{if (NR<startrow || NR>endrow)}' nagiosconfigfile.cfg > newconfigfile.cfg

    Do I have to go through this or is there a way to edit the file inline like you can with sed?
  4. Lastly, is there a better way to do this?
# 2  
Old 10-26-2010
Code:
echo `tr '\n' '|' <tst` | xargs -n10 echo | grep -v 'host_name XMPP_ec2-184-2-2-2' | tr '|' '\n'


Code:
[ctsgnb@shell ~]$ cat tst
define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-1-1-1
    alias        XMPP_ec2-184-1-1-1
    address        ip-10-10-10-10.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-2-2-2
    alias        XMPP_ec2-184-2-2-2
    address        ip-10-20-20-20.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-204-1-1-1
    alias        XMPP_ec2-204-1-1-1
    address        ip-10-30-30-30.us-west-1.compute.internal
    }
[ctsgnb@shell ~]$ echo `tr '\n' '|' <tst` | xargs -n10 echo | grep -v 'host_name XMPP_ec2-184-2-2-2' | tr '|' '\n'
define host{
 use linux-server,host-pnp
 host_name XMPP_ec2-184-1-1-1
 alias XMPP_ec2-184-1-1-1
 address ip-10-10-10-10.us-west-1.compute.internal

}

define host{
 use linux-server,host-pnp
 host_name XMPP_ec2-204-1-1-1
 alias XMPP_ec2-204-1-1-1
 address ip-10-30-30-30.us-west-1.compute.internal

}

[ctsgnb@shell ~]$

With a bit more formatting

Code:
[ctsgnb@shell ~]$ echo `tr '\n' '|' <tst` | xargs -n10 echo | grep -v 'host_name XMPP_ec2-184-2-2-2' | tr '|' '\n' | grep -vE '^$' | sed -e 's/^ /      /'
define host{
        use linux-server,host-pnp
        host_name XMPP_ec2-184-1-1-1
        alias XMPP_ec2-184-1-1-1
        address ip-10-10-10-10.us-west-1.compute.internal
}
define host{
        use linux-server,host-pnp
        host_name XMPP_ec2-204-1-1-1
        alias XMPP_ec2-204-1-1-1
        address ip-10-30-30-30.us-west-1.compute.internal
}
[ctsgnb@shell ~]$

Last edited by ctsgnb; 10-26-2010 at 09:02 PM..
# 3  
Old 10-26-2010
Quote:
Originally Posted by mglenney
My questions are:
  1. I have to put in "if (/^\t\}/ && foundblock == 1)" twice because I want it to output the line numbers and also set "foundblock=0". How can I perform multiple actions if the condition is true?
  2. or, since I'm confident the host will only be defined once, how do I output and stop parsing the file so I don't have to reset "foundblock" to 0?
  3. I plan on using "set --" to output that 8 and 14 to 2 variables within a bash script. Then I will go back through the file and output all lines but those with:
    Code:
    awk -v startrow=$start -v endrow=$end 
'{if (NR<startrow || NR>endrow)}' nagiosconfigfile.cfg > newconfigfile.cfg

    Do I have to go through this or is there a way to edit the file inline like you can with sed?
  4. Lastly, is there a better way to do this?
Use curly braces to group a set of statements to execute when the expression evaluates to true:

Code:
if( hostname == "foo )
{
    print hostname;
    found = 1;
}

You can use the exit() function to cause an early exit.
Code:
if( hostname == "foo )
{
    print hostname;
    found=1;
    exit( 0 );              # exit good; use non-zero to exit bad
}

Finally, I'd have written the programme to do it all in one pass:
Code:
 awk -v remove=XMPP_ec2-184-2-2-2 '
        /define host/ {                 # start of new block; print last section
                if( snarfed )
                        print snarfed;

                snarfed = $0;           # start capture
                next;
        }

        /host_name/ {                   # check to see if unwanted target
                if( $2 == remove )      # if it is then
                        snarfed = "";   # unset to trash the whole block
                next;
        }
        snarfed {                       # if we are snarfing from current section
                snarfed = snarfed "\n" $0;      # add next line
                next;
        }
        END {                           # must do one last for the last buffered section
                if( snarfed )
                        print snarfed;
        }
' config-file >new-file
mv config-file config-file.bak
mv new-file config-file

You can pass the host name as a variable to make it more flexible. This also preserves lines from the origiinal file that aren't 'host definition' sections -- don't know if that's a requirement or not.

Hope this helps
# 4  
Old 10-26-2010
Code:
awk '!/XMPP_ec2-184-1-1-1/' RS= ORS="\n\n" infile

Code:
define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-2-2-2
    alias        XMPP_ec2-184-2-2-2
    address        ip-10-20-20-20.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-204-1-1-1
    alias        XMPP_ec2-204-1-1-1
    address        ip-10-30-30-30.us-west-1.compute.internal
    }

These 2 Users Gave Thanks to Scrutinizer For This Post:
ctsgnb mglenney
# 5  
Old 10-26-2010
Quote:
Originally Posted by agama
Use curly braces to group a set of statements to execute when the expression evaluates to true:
Ok, thanks. That was making me nuts.

Quote:
[code]
awk -v remove=XMPP_ec2-184-2-2-2 '
/define host/ { # start of new block; print last section
if( snarfed )
print snarfed;

snarfed = $0; # start capture
next;
}

/host_name/ { # check to see if unwanted target
if( $2 == remove ) # if it is then
snarfed = ""; # unset to trash the whole block
next;
}
snarfed { # if we are snarfing from current section
snarfed = snarfed "\n" $0; # add next line
next;
}
END { # must do one last for the last buffered section
if( snarfed )
print snarfed;
}
' config-file
Cool. I knew there was a way to do this without 2 awk statements. What you wrote here is almost what I need. It was removing the "host_name" record from all the host definitions so I removed the "next;" line from the /host_name/ section.

Your comment about other text in the file made me realize I need to support that. I added in some text and any text before the first "define host" is not output. Here's the test file:

Code:
$ cat testdata.cfg 
This is some text
#this is some commented text

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-1-1-1
    alias        XMPP_ec2-184-1-1-1
    address        ip-10-10-10-10.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-2-2-2
    alias        XMPP_ec2-184-2-2-2
    address        ip-10-20-20-20.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-204-1-1-1
    alias        XMPP_ec2-204-1-1-1
    address        ip-10-30-30-30.us-west-1.compute.internal
    }

This is some after text

And this is the awk:
Code:
 awk -v remove=XMPP_ec2-184-2-2-2 '
        /define host/ {
                if( snarfed )
                        print snarfed;

                snarfed = $0;
                next;
        }

        /host_name/ {
                if( $2 == remove )
                        snarfed = "";
        }
        snarfed {
                snarfed = snarfed "\n" $0;
                next;
        }
        END {
                if( snarfed )
                        print snarfed;
        }
' testdata.cfg

and this is the result:
Code:
define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-184-1-1-1
    alias        XMPP_ec2-184-1-1-1
    address        ip-10-10-10-10.us-west-1.compute.internal
    }

define host{
    use        linux-server,host-pnp
    host_name    XMPP_ec2-204-1-1-1
    alias        XMPP_ec2-204-1-1-1
    address        ip-10-30-30-30.us-west-1.compute.internal
    }

This is some after text

  1. Any idea why the text before the first host definition gets snipped?
  2. How does this work? Is 'snarfed' a function?
# 6  
Old 10-26-2010
@Scruti

Nice one ! Lol
# 7  
Old 10-26-2010
Quote:
Originally Posted by Scrutinizer
awk '!/XMPP_ec2-184-1-1-1/' RS= ORS="\n\n" infile
Ok. Now that's awesome. Multi-line record handling. Very smart. I also had no idea that setting RS to null would separate records at blank lines. Very cool. I put those blank lines in so it would look pleasing if I had to manually modify it. Guess I got lucky Smilie

I modified it to make sure it only matches that string if it's a defined "host_name":
Code:
awk '!/host_name\tXMPP_ec2-184-1-1-1/' RS= ORS="\n\n" infile

and it works perfectly.

Thanks!!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove multiple lines from a text file

Hi I have a text file named main.txt with 10,000 lines. I have another file with a list of line numbers (around 1000) of the lines to be deleted from main.txt file. I tried with sed but it removes only a range of line numbers. Thanks for any help!! (1 Reply)
Discussion started by: prvnrk
1 Replies

2. Shell Programming and Scripting

Inline edit using sed / awk

Hi, I have file with all the lines as following format <namebindings:StringNameSpaceBinding xmi:id="StringNameSpaceBinding" name="ENV_CONFIG_PATH" nameInNameSpace="COMP/HOD/MYSTR/BACKOFFICE/ENV_CONFIG_PATH" stringToBind="test"/> I want to replace (all the lines) value of... (8 Replies)
Discussion started by: shuklaa02
8 Replies

3. Shell Programming and Scripting

Filter or remove duplicate block of text without distinguishing marks or fields

Hello, Although I have found similar questions, I could not find advice that could help with our problem. The issue: We have several hundreds text files containing repeated blocks of text (I guess back at the time they were prepared like that to optmize printing). The block of texts... (13 Replies)
Discussion started by: samask
13 Replies

4. Shell Programming and Scripting

Remove a block of Text at regular intervals

Hello all, I have a text files that consists of blocks of text. Each block of text represents a set of Cartesian coordinates for a molecule. Each block of text starts with a line that has a only a number, which is equal to the total number of atoms in the molecule. After this number is a line... (15 Replies)
Discussion started by: marcozd
15 Replies

5. Shell Programming and Scripting

How to get awk to edit in place and join all lines in text file

Hi, I lack the utter fundamentals on how to craft an awk script. I have hundreds of text files that were mangled by .doc format so all the lines are broken up so I need to join all of the lines of text into a single line. Normally I use vim command "ggVGJ" to join all lines but with so many... (3 Replies)
Discussion started by: n00ti
3 Replies

6. Shell Programming and Scripting

Replace text block in multiple files

I need to replace (delete) a text block in a bunch of files, its a html table, almost at the end of pages but the location varies. In Windows I used Filemonkey, but nothing like that in Unix? There is replace from mysql, but how does it deal with newlines? sed only works with single lines,... (6 Replies)
Discussion started by: eiland
6 Replies

7. Shell Programming and Scripting

Need to remove multiple text from a single file

Dear all, I have a file which have let us say records from A-Z. Now I want to remove multiple letter from this file using single command.. let us say I want to remove A,F,K,Y,U,P,B,S,D. I can use grep -v command but for this case i need to rerun the file several time i wana avoid using... (3 Replies)
Discussion started by: jojo123
3 Replies

8. Shell Programming and Scripting

read list of filenames from text file and remove these files in multiple directories

I have a large list of filenames from an Excel sheet, which I then translate into a simple text file. I'd like to use this list, which contains various file extensions , to archive these files and then remove them recursively through multiple directories and subdirectories. So far, it looks like... (5 Replies)
Discussion started by: fxvisions
5 Replies

9. Shell Programming and Scripting

sed / awk - inplace or inline edit

I need to remove the '&' from a file. In each line of the file, the fields are separated by ^K. I only want to remove '&' if it exists in field number 9. (example of field 9: abc&xyz) I need to do an in place/in line edit. So far I have accomplished the following: awk -F '^K' '{print... (6 Replies)
Discussion started by: hemangjani
6 Replies

10. AIX

Script to perform some actions on multiple files

I have this Korn script that I wrote (with some help) that is run by cron. I basically watches a file system for a specific filename to be uploaded (via FTP), checks to make sure that the file is no longer being uploaded (by checking the files size), then runs a series of other scripts. The... (2 Replies)
Discussion started by: heprox
2 Replies
Login or Register to Ask a Question