Omitting sections of file that contain word


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Omitting sections of file that contain word
# 1  
Old 10-02-2012
Omitting sections of file that contain word

I have a configuration file that contains hundreds of these chunks. Each "chunk" is the section that begins with "define service {" and ends with "}".


Code:
define service {
        check_command                   check_proc!java
        hostgroup_name
        service_description             Check Java
        use                             process-service
}

define service {
        check_command                   check_proc!apache
        hostgroup_name
        service_description             Check Apache
        use                             process-service
}

define service {
        check_command                   check_proc!syslog
        hostgroup_name
        host_name                        sky-01.smart.net,sky-02.smart.net
        service_description             Check Syslog
        use                             process-service
}

Now, let's assume the above configuration is from a file called process.cfg.

Now, what I wish to do is to "cat" the process.cfg file, and then omit every chunk in that file that has nothing set for the "hostgroup_name". Then redirect results to a new file. this new file which will be void of the chunks that do not have anything set for hostgroup_name.

There is a challenge to this.

If you look at each chunk above, you will see that all of them have nothing set for "hostgroup_name". So if i cat the process.cfg and I run an awk code against it, what would be left will be, most likely, the last chunk.

If you look at the very last chunk, you see that, there are hosts set up for it. the "host_name" section has hosts defined. so, in this case, i dont want this chunk to be omitted. i would ever like to have it ommitted if the host_name field exist, AND is also empty as well.

os: linux
shell: bash

sample of a code that can be modified to do the above:
Code:
cat process.cfg | awk 'NR==FNR {a=(a?a"|":"")$0; next}; {gsub(a,""); gsub(" ,+"," "); gsub(",+$",""); gsub(",+",",")} !/hostgroup_name[ \t\n]*use/ {print $0 RS}' /tmp/listofhostgroups.txt RS="}\n" > newfile

the "/tmp/listofhosts.txt" may not be necessary. but i dont know how to make this code woork.

any ideas?
# 2  
Old 10-02-2012
That is a useless use of cat. awk, or any other shell command, does not need cat's help to read a single file.

If you set awk's record separator to blank, it will take blank lines to be the separators between records here:

Code:
awk -v RS="" -F"\n" '{ P=1
        for(N=1; N<=NF; N++) if($N ~ /hostgroup_name[ \t]*$/) P=0 } P' inputfile

This User Gave Thanks to Corona688 For This Post:
# 3  
Old 10-02-2012
Quote:
Originally Posted by Corona688
That is a useless use of cat. awk, or any other shell command, does not need cat's help to read a single file.

If you set awk's record separator to blank, it will take blank lines to be the separators between records here:

Code:
awk -v RS="" -F"\n" '{ P=1
        for(N=1; N<=NF; N++) if($N ~ /hostgroup_name[ \t]*$/) P=0 } P' inputfile

this seems to work marvelously. the problem is, it also removes chunks that has "host_name".

if the code you provided finds a section that looks something like this:
Code:
define service {
        check_command                   check_proc!syslog
        hostgroup_name
        host_name                        sky-01.smart.net,sky-02.smart.net
        service_description             Check Syslog
        use                             process-service
}

i want it to only delete the line that has the "hostgroup_name" on it. meaning, if hostgroup_name is empty, but the host_name is not, then, delete line containing hostgroup_name and still output the chunk (but with the hostgroup_name line deleted).

however, if both hostgroup_name AND host_name are empty, then OMIT the entire chunk.

there will be chunks where "hostgroup_name" is defined, but "host_name" is not. and vice versa.
# 4  
Old 10-02-2012
Code:
awk -v RS="" -v ORS="\n\n" -F"\n" '{ P=0
        for(N=1; N<=NF; N++) if($N ~ /host(group)?_name[ \t]+[^ \t]/) P=1 } P' inputfile

# 5  
Old 10-02-2012
Quote:
Originally Posted by Corona688
Code:
awk -v RS="" -v ORS="\n\n" -F"\n" '{ P=0
        for(N=1; N<=NF; N++) if($N ~ /host(group)?_name[ \t]+[^ \t]/) P=1 } P' inputfile


i'm sorry. when i said:

Code:
there will be chunks where "hostgroup_name" is defined, but "host_name" is not. and vice versa.

i meant to say, while there are some chunks where hostgroup_name is defined, but "host_name" is not, and vice versa, there are also some chunks where neither are present. meaning, there are junks where lines for "host_name" and "hostgroup_name" do not exist.

so this code u just pasted didn't take that into account. in the final file that was outputted, those chunks were removed (the chunks that did not have lines with the word hostgroup_name OR host_name on them). i dont want that.

i'm very sorry for the confusion. can you please modify your code to take that into account?

thank you!
# 6  
Old 10-03-2012
So, if a block contains neither host_name or hostgroup_name, it should be printed? But if that information is present and blank, it should not?

Code:
awk -v RS="" -v ORS="\n\n" -F"\n" '{ P=0; R=0
        for(N=1; N<=NF; N++)
        {
                if($N ~ /host(group)?_name/) R++
                if($N ~ /host(group)?_name[ \t]+[^ \t]/) P++;
        }
        } P || (!R)' inputfile

This User Gave Thanks to Corona688 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk removing sections of a file

I have a file that looks liek this (see below). can somebody provide me with and awk or sed command that can take a piece of the file starting from the time to the blank line and put in into another file. For example: How would I get the data from 10:56:11 to the blank line. Two things: ... (5 Replies)
Discussion started by: BeefStu
5 Replies

2. Programming

extract different sections of a file

Hi All, I have a file with the data 10;20;30;40;50;60;70;80;123;145;156;345. the output i want is the first fourth sixth elements and everything from there on. How do i achieve this. (1 Reply)
Discussion started by: raghu_shekar
1 Replies

3. Shell Programming and Scripting

How to edit file sections that cross multiple lines?

Hello, I'm wondering where I could go to learn how to edit file sections that cross multiple lines. I'm wanting to write scripts that will add Gnome menu entries for all users on a system for scripts I write, etc. I can search an replace simple examples with sed, but this seems more complex. ... (8 Replies)
Discussion started by: Narnie
8 Replies

4. Shell Programming and Scripting

Modify sections of the line in a file

Hello.. I have a line in a file which I have to edit: the line looks like: <!]> Sometimes, the section of the line can have only one entry for cn, or maybe more than 2 like below: <!]> I have a variable which has the following value: CN="(cn=MNO)(cn=XYZ)" I need to replace the part... (4 Replies)
Discussion started by: chiru_h
4 Replies

5. Shell Programming and Scripting

Remove sections of a xml file with sed

I've been trying to remove some lines of a xml file that looks like this: <parent> <child>name1</child> <lots_of_other tags></lots_of_other_tags> </parent> <parent> <child>name2</child> <lots_of_other tags></lots_of_other_tags> </parent> <parent> <child>name3</child> ... (5 Replies)
Discussion started by: viniciusandre
5 Replies

6. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies

7. Shell Programming and Scripting

Best way to remove sections of text from a file

Greetings! I found this fourm via a google search on "sed expressions". I have a file that contains notices and they are all the same length in lines. For example the file would contains 15 notices, each being 26 lines each. I need some way to eliminate notices that contain a "S" in a particular... (8 Replies)
Discussion started by: cals64
8 Replies

8. Shell Programming and Scripting

extract multiple sections of file

I have a file that I need to parse multiple sections from the file. The file contains multiple lines that start with ST (Abunch of data) Then the file contains multiple lines that start with SE (Abunch of data) SE*30*0001 ST*810*0002 I need all of the lines between and including these.... (6 Replies)
Discussion started by: rgentis
6 Replies

9. UNIX for Advanced & Expert Users

extract multiple sections of a file

I have a file that I need to parse multiple sections from the file. The file contains multiple lines that start with ST (Abunch of data) Then the file contains multiple lines that start with SE (Abunch of data) SE*30*0001 ... (1 Reply)
Discussion started by: rgentis
1 Replies

10. Shell Programming and Scripting

omitting lines from file A that are in file B

I've got file A with (say) 1M lines in it ... ascii text, space delimited ... I've got file B with (say) 10M lines in it ... same structure. I want to remove any lines from A that appear (identically) in B and print the remaining (say) 900K lines. (And I want to do it in zero time of... (14 Replies)
Discussion started by: gneen
14 Replies
Login or Register to Ask a Question