Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Parse configuration file & add line in particular section

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parse configuration file & add line in particular section
# 1  
Old 03-17-2011
Parse configuration file & add/remove line in particular section
Greetings,

I recently built a replicated DRBD, Heartbeat, & iSCSI Target Initiator storage server on Ubuntu 10.04 to offer shared storage to server Vmware ESX and Microsoft Clusters. Everything works flawlessly, however I wanted to make a script to create, remove, grow volumes to offer ESX rather then having a technical diagram and instructions so that other technicians are able to do this process with limited Linux knowledge.

I know the basis to Linux scripting with variables and command scripting but I am not sure how to approach parsing a configuration file for iSCSI Target Initiator and adding lines between existing lines under a sub group. I figure it has to be done in an array but I am not sure how to keep it within a parameter as well as add a line adding the next sequential number available.

Let me give an example of my configuration file called ietd.conf:

Code:
Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
	IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
	IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
	IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
	Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
	IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume3,Type=blockio
	Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

What I need to do is when my script asks the user which Server they wish to add storage too (lets say ESX) that it targets “Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1” and reads each “Lun #.....” and adds a line below LUN 2 (in example configuration file) as LUN 3? I will replace “/dev/replicated/<volume>” with a variable which I can do. I just do not know how to parse only the lines between “Target iqn” and the next “Target ign” looking for “LUN #” and adding a new line below the existing LUN with a higher sequential number.

I am also not sure how to approach gaps in LUN numbers when volumes are deleted. My assumption is to take the highest LUN # under the target and add one to it but I am not sure if this is correct?

Any insight would be great. Thanks.
Last edited by Aeudian; 03-22-2011 at 12:24 PM.. Reason: Title Change
# 2  
Old 03-17-2011
How about this?
Code:
#!/bin/bash

servername=Microsoft.Storage.1 #to find appropriate paragraph

nums=( $( awk '
/'"$servername"'/,/^\s*$/  { #process only between $servername and empty line
  if(/Lun/){numLun=$2; numRec=FNR} #if found line with 'Lun', capture 2nd field and line number
}
END{
print numLun" "numRec
} ' ietd.conf ) ) 
#nums is an array; first element is the field right after 'Lun'
#and second element is its line number 

newEntry="Lun $((${nums[0]}+1)) and whatever else you enter" #what should be inserted

sed "${nums[1]} a \\\t$newEntry" ietd.conf   #'a' command (append) to the appropriate line

Last edited by mirni; 03-17-2011 at 11:02 PM.. Reason: inserting comments
This User Gave Thanks to mirni For This Post:
Aeudian
# 3  
Old 03-22-2011
Thank you! I was able to utilize what you gave me with variables to make it work in my script perfectly.

I am working on my delete part of my script where now I am looking to gather all the LUN #'s under a particular field from the ietd.conf file (such as "Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1" and offer them to the user as a selection choice such as:

1) Lun 0 Path=/dev/replicated/resources,Type=blockio
2) Lun 1 Path=/dev/replicated/volume1,Type=blockio
3) Lun 2 Path=/dev/replicated/volume2,Type=blockio

So when the user selects 2, the line "Lun 1 Path=/dev/replicated/volume1,Type=blockio" is removed from the file. Any advice?
# 4  
Old 03-22-2011
Could this help you ?
Code:
#!/usr/bin/perl

use strict;
my ($server,@flds,$answer,$i,@path,$ret);
system("clear");
{
local $/="\n\n";
$server='Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1';
open(FH,"ietd.conf") or die "$!\n";
while (<FH>){
chomp;
if(/$server/) {
@flds=map("$_\n",grep(/Lun/,split(/\n/)));
foreach (@flds) {s/^\s+//g; print ++$i.") ".$_;}
}
}
}
close(FH);
printf "Enter option:";
$answer=<STDIN>;
chomp;
die  "Invalid Option\n" if ($answer <= 0 ||  $answer > $i);
open(FH,"ietd.conf") or die "$!\n";
{
        local $/="\n\n";
        while(<FH>){
        if(/$server/) {
        @path=split(/=/,$flds[$answer-1]);
        $path[1]=~s/,.*$//g;
        system("./pass_variable.sh $path[1]");
        s/$flds[$answer-1]\s*//g;
        } print $_;
        }

}
close(FH);

Code:
#cat pass_variable.sh
#!/bin/sh
echo ""
echo "Bash script start"
echo $1
echo "Bash script end"
sleep 2

Please make bash script executable using
Code:
chmod +x pass_variable.sh

Last edited by pravin27; 03-22-2011 at 02:46 PM..
# 5  
Old 03-22-2011
Thank you.

I have two questions as I am not familiar with the perl programming language.

1) When I execute what you gave me against the ietd.conf file I am prompted to "Enter option:" but nothing is printed on the screen telling me the available options or lines that it has found. No matter what I type I get the die "Invalid Option" and exit.

2) My current script is built in /bin/bash can I make perl and bash work together as my future plan is it extract Path="/dev/replicated/<name>" into a variable of the line being removed from ietd.conf for my second stage of deleting the lvm.
# 6  
Old 03-22-2011
It's working fine at my end see the below o/p. I have modified my perl code in my previous post.You can see how to pass variable to bash script from perl.
ietd.conf file
Code:
#cat ietd.conf
Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
        IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
        IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
        IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
        Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
        IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Perl script invocation
Code:
#perl DeleteLines.pl

Output
Code:
1) Lun 0 Path=/dev/replicated/resources,Type=blockio
2) Lun 1 Path=/dev/replicated/volume1,Type=blockio
3) Lun 2 Path=/dev/replicated/volume2,Type=blockio
Enter option:2

Bash script start
/dev/replicated/volume1
Bash script end
Target iqn.2011-03.stroageserver.workgroup.net:VMware.Storage.1
        IncomingUser iqn.1998-01.com.vmware:esx001-3c494e40 password
        IncomingUser iqn.1998-01.com.vmware:esx002-7e5d5341 password
        IncomingUser iqn.1998-01.com.vmware:esx003-76a022ad password
        Lun 0 Path=/dev/replicated/resources,Type=blockio
        Lun 2 Path=/dev/replicated/volume2,Type=blockio
        Alias VMwareStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

Target iqn.2011-03.stroageserver.workgroup.net:Microsoft.Storage.1
        IncomingUser MSiSCSIAdmin password
        Lun 1 Path=/dev/replicated/volume1,Type=blockio
        Alias MicrosoftStorageArray
        MaxConnections         1
        InitialR2T             Yes
        ImmediateData          No
        MaxRecvDataSegmentLength 8192
        MaxXmitDataSegmentLength 8192
        MaxBurstLength         262144
        FirstBurstLength       65536
        DefaultTime2Wait       2
        DefaultTime2Retain     20
        MaxOutstandingR2T      8
        DataPDUInOrder         Yes
        DataSequenceInOrder    Yes
        ErrorRecoveryLevel     0
        HeaderDigest           CRC32C,None
        DataDigest             CRC32C

This User Gave Thanks to pravin27 For This Post:
Aeudian
# 7  
Old 03-22-2011
Thanks you for your assistance, I really do appreciate it. Everything seems to be working flawlessly.

Thanks again.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to change file section into each line?

Hi Gurus, I have below file which has different sections, need to move the sections to beginning of the each record. original file aaa bbb ccc ddd eee fff output file. aaa bbb ccc ddd eee fff (6 Replies)
Discussion started by: green_k
6 Replies

2. Shell Programming and Scripting

awk to parse section of csv into array

In the awk below I am trying to parse the Sample Name below the section. The values that are extracted are read into array s(each value in a row seperated by a space) which will be used later in a bash script. The awk does execute but no values are printed. I am also not sure how to print in a row... (1 Reply)
Discussion started by: cmccabe
1 Replies

3. Shell Programming and Scripting

Grep or print each section of a file on one line with a separator

I can obtain information from itdt inventory command however it display as below, I'd like to print each entity on one line but seperated by : the file is something like and each section ends with Volume Tag Drive Address 256 Drive State ................... Normal ASC/ASCQ... (3 Replies)
Discussion started by: gefa
3 Replies

4. Shell Programming and Scripting

Prepend first line of section to each line until the next section header

I have searched in a variety of ways in a variety of places but have come up empty. I would like to prepend a portion of a section header to each following line until the next section header. I have been using sed for most things up until now but I'd go for a solution in just about anything--... (7 Replies)
Discussion started by: pagrus
7 Replies

5. UNIX for Dummies Questions & Answers

Line & File Manipulation - add spaces between characters

Is there an awk, sed, vi or any line command that adds Field Separators (default spaces) to each line in a file? $cat RegionalData 12FC2525MZLP8266900216 12FC2525MZLP8266900216 12FC2525NBLP8276900216 12FC2525NBLP8276900216 Desired results: 1 2 F C 2525 MZ LP 826 690 02 16 1 2 F C... (2 Replies)
Discussion started by: MS75001
2 Replies

6. Shell Programming and Scripting

Parse Multi-Section Configuration File

Hello all, Sample configuration file: username = root password = admin IpAddress = 192.168.2.90 HttpCommand = /getfile?hello.jpg Username = root2 Password = admin2 Passive = no Host = 192.168.1.100 Path = /uploads Username = root3 Password = adming Passive = no Host =... (10 Replies)
Discussion started by: LAVco
10 Replies

7. Shell Programming and Scripting

Search and extract by section from configuration

Hi, I understand either AWK or SED can do this, but I not sure how to extract the following configuration in section. Meaning when I need to find code with " ip helper-address 192.168.11.2" , it would start from "interface Serial0/0" and "interface FastEthernet0/1". Only displaying both section... (2 Replies)
Discussion started by: haphazard
2 Replies

8. Shell Programming and Scripting

SED help (remove line::parse again::add line)

Aloha! I have just over 1k of users that have permissions that they shouldn't under our system. I need to parse a provided list of usernames, check their permissions file, and strip the permissions that they are not allowed to have. If upon the permissions strip they are left with no permissions,... (6 Replies)
Discussion started by: Malumake
6 Replies

9. Shell Programming and Scripting

sed & awk--get section of file based 2 params

I need to get a section of a file based on 2 params. I want the part of the file between param 1 & 2. I have tried a bunch of ways and just can't seem to get it right. Can someone please help me out.....its much appreciated. Here is what I have found that looks like what I want....but doesn't... (12 Replies)
Discussion started by: Andy Cook
12 Replies

10. UNIX for Dummies Questions & Answers

help find a section line of a file

hi, I have a 20 line file. I need a command which will brinf back a specific line based upon the line number I enter. e.g. the file looks like this and is called file1 jim is a man john is a woman james is a man wendy is a woman lesley is a woman i want a command that will... (4 Replies)
Discussion started by: sureshy
4 Replies
Login or Register to Ask a Question