SED 4.1.4 - INI File Change Problem in Variables= in Specific [Sections] (Guru Help)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting SED 4.1.4 - INI File Change Problem in Variables= in Specific [Sections] (Guru Help)
# 1  
Old 06-01-2009
Question SED 4.1.4 - INI File Change Problem in Variables= in Specific [Sections] (Guru Help)

GNU sed version 4.1.4 on Windows XP SP3 from GnuWin32

I think that I've come across a seemingly simple text file change problem on a INI formatted file that I can't do with SED without side effects edge cases biting me. I've tried to think of various ways of doing this elegantly and quickly with SED but it seems to me that I'm going to have to try with AWK or finally learn PERL and move to it.

However, before I take these drastic measures let me present the problem here to the SED gurus.

The problem basically stems from the fact that INI format text files do not have a closing tag for the end of the [section], the start of the next section or end of file is the end of the section. This prevents me from using the AWK address range "/ADDR1/,/ADDR2/" feature for limiting the edits to only the section because of the annoying side effect that is the /ADDR2/ range is "inclusive" as per the SED info - 3.2 Selecting lines with `sed' page.

This /ADDR2/ inclusiveness causes problems since once it hits for the next section start then /ADDR1/ doesn't hit again so code for that address range is skipped entirely for the next section causing the SED script to hop-scotch across the INI file skipping every other match eligible section.

Here is an actual sample INI file to work on as an example of the format in case someone is unfamiliar with it. The idea is to change the values in the [supplies_res_*] sections without touching the [supplies_generic] or [trade_generic_sell] sections.

INI File

Code:
[supplies_generic]
ammo_12x70_buck            = 3,    1
ammo_12x76_zhekan        = 3,    0.5

ammo_9x18_fmj            = 2,    1


[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 3, 1

wpn_ak74            = 1, 0.3



[supplies_res_6]:supplies_res_3
ammo_12x76_dart            = 3, 0.5

medkit_army            = 5, 1


[trade_generic_sell]
ammo_9x18_fmj            = 1.5, 1.5
ammo_9x19_fmj            = 1.5, 1.5

ammo_11.43x23_fmj        = 1.5, 1.5

ammo_12x70_buck            = 1.5, 1.5
ammo_12x76_zhekan        = 1.5, 1.5

ammo_5.45x39_fmj        = 1.5, 1.5

ammo_5.56x45_ss190        = 1.5, 1.5

grenade_rgd5            = 1.5, 1.5




The code below shows the problem with trying to create the address range using the beginning of the next section as the end range, since the inclusiveness of /ADDR2/ address makes the code skip any changes in that section.

SED Code Example - Broken = Hop-Scotch Section Skipping

Code:
# Section Address Range
/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I , /^[[:space:]]*\[/


If I try other /ADDR2/ ranges such as "$" for end of file then all sections that contain the same variable will be changed to the end of the file, not something that I want. I cannot use blank line /^[[:space:]]*$/ for ADDR2 since the sections have blank spaces and lines inside them.

If I try to get really clever and do something like below then it doesn't work either no matter what I try to do or think of.


SED Code Example - Broken = Manual Line Reading Doesn't Work (Late Night Idea)

Code:
# INI Section
/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I {

	# Read Line
	: read;
	n;

	# Next Section Jump Out
	/^[[:space:]]*\[[[:space:]]*%_section%[[:space:]]*\]/I! {
		/^[[:space:]]*\[/ b;
	};

	# Variable Line
	/^[[:space:]]*%_variable%[[:space:]]*=/I {

		# Value Replace
		# Note: Retain same whitespace after replacement except for value.
		s/^([[:space:]]*%_variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%_value%/I;

	};

	# Loop
	b read;

};

# 2  
Old 06-01-2009
Can you explain what you're trying to replace and post the expected output within code tags?
# 3  
Old 06-01-2009
@OP, sed can definitely operate and edit a file, however, in your case, its not a suitable tool. You should, if possible, try to use available packages for parsing ini files with languages like Python or Perl. It will makes your sysadmin life easier. An example with Python
Code:
#!/usr/bin/env python
import ConfigParser
config = ConfigParser.ConfigParser()
config.read("file")
for section in config.sections():    
    print "SEction: [%s]" % section
    for options in config.options(section):            
        print "%s = %s" % (options, config.get(section, options))

output
Code:
# ./test.py
SEction: [trade_generic_sell]
grenade_rgd5 = 1.5, 1.5;x;xxx
ammo_9x19_fmj = 1.5, 1.5
ammo_9x18_fmj = 1.5, 1.5
ammo_11.43x23_fmj = 1.5, 1.5
ammo_12x76_zhekan = 1.5, 1.5
ammo_5.45x39_fmj = 1.5, 1.5
ammo_5.56x45_ss190 = 1.5, 1.5
ammo_12x70_buck = 1.5, 1.5
SEction: [supplies_res_3]
ammo_12x76_zhekan = 3, 1
wpn_ak74 = 1, 0.3
SEction: [supplies_generic]
ammo_12x76_zhekan = 3,    0.5
ammo_12x70_buck = 3,    1
ammo_9x18_fmj = 2,    1
SEction: [supplies_res_6]
medkit_army = 5, 1
ammo_12x76_dart = 3, 0.5

the above just prints out the values of the ini files, however, to change any value is just trivial.
anyway, its your call whether to use other tools besides sed or not.
# 4  
Old 06-01-2009
Quote:
Originally Posted by Franklin52
Can you explain what you're trying to replace and post the expected output within code tags?
Below is an example. Only change the values for the variables inside the supplies_res_* section without touching anything else.

INI Before Change

Code:
[supplies_generic]
ammo_12x76_zhekan        = 3,    0.5

[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 3, 1

[supplies_res_6]:supplies_res_3
ammo_12x76_zhekan        = 3,    0.5

[trade_generic_sell]
ammo_12x76_zhekan        = 1.5, 1.5



INI After Change

Code:
[supplies_generic]
ammo_12x76_zhekan        = 3,    0.5

[supplies_res_3]:supplies_generic
ammo_12x76_zhekan        = 100, 1.0

[supplies_res_6]:supplies_res_3
ammo_12x76_zhekan        = 100, 1.0

[trade_generic_sell]
ammo_12x76_zhekan        = 1.5, 1.5

Sounds simple, like something that you can do with SED address ranges but it seems really hard or impossible.

Quote:
Originally Posted by ghostdog74
@OP, sed can definitely operate and edit a file, however, in your case, its not a suitable tool. You should, if possible, try to use available packages for parsing ini files with languages like Python or Perl. It will makes your sysadmin life easier.
You are correct that another scripting tool would make it easier and that using a dedicated INI file parser library would be the best solution. I think that this is the way to go at this point because I can't find a simple solution.

I chose SED because that is the tool that I know how to use and I thought it would be easy to do this with a single line of code for SED.

SED Script - One Line INI Variable Change in Section (Broken!, skips every other section when consecutive sections match)

Code:
/^\[%section%\]/I, /^\[/ s/^(%variable%=)(.*)$/\1%value%/I;

Whitespace Robust

Code:
/^[[:space:]]*\[[[:space:]]*%section%[[:space:]]*\]/I, /^[[:space:]]*\[/ s/^([[:space:]]*%variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%value%/I;




SED Negative RegEx Modifier ! - Not The Solution

SED has a unary NOT "!" modifier available but it only applies to the entire address, singleton and range, and not for individual RegExp in the address. This modifier cannot be used for this solution because the match for ADDR2 has to be a positive match for the start of the next section but once you get a positive match then this line becomes inclusive in the address.



SED NonInclusive ADDR2 Modifier Required - New Feature Request

The real solution would be SED to have an option or a modifier available for the ADDR2 so that when it matches that line it becomes non-inclusive. This way address ranges for SED could be used to parse files that have a structure with markings only delimiting the beginning of sections but not the ending of sections.

If you think about it most of the human designed data formats for ad-hoc purposes such as documents or simple data files only delineate the beginning of a new section but never add any markings for the ending since they treat the beginning of a new section as that marking.

I think that this improvement to SED would be a worth while endeavor to include in the program. The problem is that I am not a programmer and I don't know how SED handles the stream input so I don't know how difficult it would be for ADDR2 to be made non-inclusive.

It could be simple with the internal line counter being made to not increment on ADDR2 match so that the next cycle of the script hits ADDR2 line again. But this could create a problem with stream processing or file editing since it is customary for SED to only hit each line once and not repeat it.

However, I always thought that the inclusiveness of ADDR2 was a weakness in the address range design since this is not the first time that I butted my head against this wall when trying to do seemingly simple file changes with SED only to realize that what I want to do just cannot be done, even if I go into SED Guru Mode and start doing branches with b, t, T.

After I hit this realization I usually boink myself and try to go next level up to AWK while silently cursing in my breath for never learning PERL in the first place to move all my file and line editing in it.

Last edited by JakFrost; 06-01-2009 at 01:20 PM..
# 5  
Old 06-01-2009
You can try this with awk. Suppose you want to change the value of the section [supplies_res_6] of ammo_12x76_zhekan with 100, 1.0:

Code:
awk -F":| " -v var="6" -v v1="100, 1.0" -v opt="ammo_12x76_zhekan" '
$1 == "[supplies_res_"var"]" && !f {f=1}
f && $1 == opt{sub("=.*","= "v1);f=0}
1' file

# 6  
Old 06-03-2009
Bug

Quote:
Originally Posted by Franklin52
You can try this with awk. Suppose you want to change the value of the section [supplies_res_6] of ammo_12x76_zhekan with 100, 1.0:
Thanks for the code but I won't be using it or needing it since I found the solution for SED. Your messy code reminded me of something that I did with SED a few years back when I had a similar problem. Your usage of the hard to understand cryptic "f=0" and "f=1" variable reminded me that. Thanks for the reminder though.



SED Solution Remembered and Found! - Hold Space As Variable

The idea is to use the hold space as a boolean variable to hold the line of the section that you are currently inside and then when you find the field in the file that you want to change, you check if you are in the proper section before making any changes to the field.

I used this with a previous problem and I just realized that I had the solution below. This works great and the code is below.

Code:
# Section Find and Save
/^[[:space:]]*\[/ h;

# Variable Line
/^[[:space:]]*%variable%[[:space:]]*=/I {

	# Section Recall
	x;

	# Section Check
	/^[[:space:]]*\[[[:space:]]*%section%[[:space:]]*\]/I {

		# Line Recall
		x;

		# Value Replace
		# Note: Retain same whitespace after replacement except for value.
		s/^([[:space:]]*%variable%[[:space:]]*=[[:space:]]*)(.*)[[:space:]]*$/\1%value%/I;

		# Section Recall
		x;
	};

	# Line Recall
	x;

};

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Using sed to change values after a specific string

Hello I have a script that searches a file for a specific string and then changes the nth column after that string. I have searched online for how to do this with sed but have not seemed to find a solution that works for me. I am using bash. Some background info: - Currently I am using awk to... (4 Replies)
Discussion started by: prodigious8
4 Replies

2. Shell Programming and Scripting

Add character to specific columns using sed or awk and make it a permanent change

Hi, I am writing a shell script where I want that # should be added in all those lines as the first character where the pattern matches. file has lot of functions defined a.sh #!/bin/bash fn a { beautiful evening sunny day } fn b { } fn c { hello world .its a beautiful day ... (12 Replies)
Discussion started by: ashima jain
12 Replies

3. Shell Programming and Scripting

sed - String substitution within specified section in ini type file

Hello. I am trying to modify a config file which is in windows *.ini type file. I have found a piece of code here :linux - Edit file in unix using SED - Stack Overflow As I can't make it doing the job , I am trying to find a solution step by step. here a modified sample file : my_sample.ini... (1 Reply)
Discussion started by: jcdole
1 Replies

4. Shell Programming and Scripting

Trouble with sed and ini file parsing

hi people, i'm having a hard time trying to extract a list of vars delimited by section inside a ini file ... let's consider this ini file : ; config file DESC = "channel synchro TGG01" DMM_VER = DMM23 PATH_FIFO = /users/tgg00/fifo QRT = BTS01.TGG.01.2 MODE_TRACE... (5 Replies)
Discussion started by: odium74
5 Replies

5. UNIX for Dummies Questions & Answers

How to use sed to copy specific lines from a file using shell variables?

hello! I am trying to use sed to copy specific set of lines from a file for which the starting and ending line numbers of the lines to be copied are stored in shell variables. How can i copy those lines? if the input_file is something like this and if the following is the script a=2 b=4... (4 Replies)
Discussion started by: a_ba
4 Replies

6. Shell Programming and Scripting

Change specific occurence with sed

Hello, I have this file. aaa port=1234 time bbb port=2233 name ccc port=4444 name Is there any way with sed to change only the occurence of "port" which comes after section to have as output : (12 Replies)
Discussion started by: rany1
12 Replies

7. UNIX for Dummies Questions & Answers

Using SED to change a specific word's color?

Ok so all i'm trying to do here is output a file and change the color of a specific word. I can't use grep with color because I need all lines of the file not just lines that match the pattern. I can get this substitution to work but when it displays it shows exactly what i'm putting it rather... (14 Replies)
Discussion started by: MrEddy
14 Replies

8. Shell Programming and Scripting

using sed to replace a specific string on a specific line number using variables

using sed to replace a specific string on a specific line number using variables this is where i am at grep -v WARNING output | grep -v spawn | grep -v Passphrase | grep -v Authentication | grep -v '/sbin/tfadmin netguard -C'| grep -v 'NETWORK>' >> output.clean grep -n Destination... (2 Replies)
Discussion started by: todd.cutting
2 Replies

9. Shell Programming and Scripting

Remove sections of a xml file with sed

I've been trying to remove some lines of a xml file that looks like this: <parent> <child>name1</child> <lots_of_other tags></lots_of_other_tags> </parent> <parent> <child>name2</child> <lots_of_other tags></lots_of_other_tags> </parent> <parent> <child>name3</child> ... (5 Replies)
Discussion started by: viniciusandre
5 Replies

10. Shell Programming and Scripting

Parsing file, yaml file? Extracting specific sections

Here is a data file, which I believe is in YAML. I am trying to retrieve just the 'addon_domains" section, which doesnt seem to be as easy as I had originally thought. Any help on this would be greatly appreciated!! I have been trying to do this in awk and mostly bash scripting instead of perl... (3 Replies)
Discussion started by: Rhije
3 Replies
Login or Register to Ask a Question

Featured Tech Videos