Parsing a control file loop


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing a control file loop
# 1  
Old 05-11-2018
Parsing a control file loop

Hi,

Let's say I have a control file like this:

Code:
RHEL apple "echo apple"
RHEL bravo "ls -l bravo*"
RHEL church "chmod church.txt"

SUSE drive "chown user1 drive.txt"
SUSE eagle "echo "eagle flies""
SUSE feather "ls -l feather*"

HP-UX google "sed 's/^Google.*$/&\
ACTION: go to www.google.com/' google.txt"
HP-UX heartache "ls -l heartache.txt* | awk '{ print $9 }'"

...and so on.

In my main script, after I get the operating system of the machine I'm executing the script on and assigning it to variable $OPER_SYS, the script will read this control file. For every line with the value of $OPER_SYS, I want the command on the last column be executed.

Please take note of the carriage return on the first HP-UX line. It's intended, because I want the sed command to find the line starting with Google and add a new line after that with the text on the right hand side... and \n doesn't work in HP-UX sed for some reason (if you guys know a way to simplify that, please let me know).

Obviously, the script I'm creating will be executed on several servers with different platforms i.e. RHEL, SUSE, HP-UX, Solaris so I want the code to be executable on all of them (I'm having difficulties with HP-UX obviously).

Thanks in advance.
# 2  
Old 05-11-2018
Quote:
Originally Posted by The Gamemaster
In my main script, after I get the operating system of the machine I'm executing the script on and assigning it to variable $OPER_SYS, the script will read this control file. For every line with the value of $OPER_SYS, I want the command on the last column be executed.
Ok, let us start with the easy part: the most basic thing you have to take care of is the difference between these systems: HP-UX has (IIRC) a Korn Shell as default shell, some Linux systems (sorry, Linux is not my strong side) have bash, others have even more obscure shells (i remember running into some dash awhile ago, whatever that is). Furthermore the exact composition of the default PATH variable may different, i.e /bin is sometimes a link to /usr/bin, sometimes not, etc.. My suggestion is to first create a large case-switch where you take care of these differences, like this sketch:

Code:
case $OPER_SYS in
     RHEL)
          export PATH=....
          export TERM=....   # in case you need it
          ....
 
     SUSE)
          export PATH=....
          export TERM=....   # in case you need it
          ....

     HPUX)
          export PATH=....
          export TERM=....   # in case you need it
          ....

esac

...rest of your "OS-independent" code here...

Notice that i.e. HP-UX has some peculiarities: the df command exists, but if you want an output that resembles what you would expect you have to use bdf. (This might not be relevant in your case, but there might be similar things that are.) Also notice that commands dealing with package management, user management, logical volumes and similar things are different in different platforms. You may need to create a function for these things which gets some parameters and then does OS-dependent things depending on the value of $OPER_SYS.

Quote:
Originally Posted by The Gamemaster
Please take note of the carriage return on the first HP-UX line. It's intended, because I want the sed command to find the line starting with Google and add a new line after that with the text on the right hand side...
This - and similar problems with special characters - is a problem which IMHO cannot be fully solved in shell: shells always interpret in some way what they read. Masking/escaping character in your input will only get you that far. A real solution would be a parser, but it would have to be written in a HLL.

So, if you can narrow down a list of special characters you need to be able to cope with - starting with the newline - a (maybe not very pretty but workable) solution might be found, otherwise i suggest you implement your problem not in shell but in some high-level language.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 3  
Old 05-11-2018
Quote:
Originally Posted by bakunin
Ok, let us start with the easy part: the most basic thing you have to take care of is the difference between these systems: HP-UX has (IIRC) a Korn Shell as default shell, some Linux systems (sorry, Linux is not my strong side) have bash, others have even more obscure shells (i remember running into some dash awhile ago, whatever that is). Furthermore the exact composition of the default PATH variable may different, i.e /bin is sometimes a link to /usr/bin, sometimes not, etc.. My suggestion is to first create a large case-switch where you take care of these differences, like this sketch:

Code:
case $OPER_SYS in
     RHEL)
          export PATH=....
          export TERM=....   # in case you need it
          ....
 
     SUSE)
          export PATH=....
          export TERM=....   # in case you need it
          ....

     HPUX)
          export PATH=....
          export TERM=....   # in case you need it
          ....

esac

...rest of your "OS-independent" code here...

Notice that i.e. HP-UX has some peculiarities: the df command exists, but if you want an output that resembles what you would expect you have to use bdf. (This might not be relevant in your case, but there might be similar things that are.) Also notice that commands dealing with package management, user management, logical volumes and similar things are different in different platforms. You may need to create a function for these things which gets some parameters and then does OS-dependent things depending on the value of $OPER_SYS.



This - and similar problems with special characters - is a problem which IMHO cannot be fully solved in shell: shells always interpret in some way what they read. Masking/escaping character in your input will only get you that far. A real solution would be a parser, but it would have to be written in a HLL.

So, if you can narrow down a list of special characters you need to be able to cope with - starting with the newline - a (maybe not very pretty but workable) solution might be found, otherwise i suggest you implement your problem not in shell but in some high-level language.

I hope this helps.

bakunin
Thanks for the tips here. I wish I could learn a new high-level language right now (Python comes to mind), but right now it's out of the question because I have a deadline with this script. The commands I put there are just examples, but the script that I'm trying to create will update the same configuration file on all servers that we manage. So in reality, that control file will only have sed commands (or any other command that can modify the configuration file if I hit a wall using sed, like with the case with HP-UX servers). So far I'm only having a problem with that carriage return because all of the sed statements will have a new line appended to the regexp being searched. I hope I have narrowed it down to you.

Last edited by The Gamemaster; 05-11-2018 at 07:40 PM..
# 4  
Old 05-12-2018
Can anyone just answer my first question?

Quote:
In my main script, after I get the operating system of the machine I'm executing the script on and assigning it to variable $OPER_SYS, the script will read this control file. For every line with the value of $OPER_SYS, I want the command on the last column be executed.
I'll just find a solution regarding special characters somehow. Hopefully someone can help me out, I really need to finish this script this week.
# 5  
Old 05-13-2018
The commands in quotes are troublesome.
Better have a tag that indicates where the (unquoted) command ends.
E.g. an empty line could do it.

Or perhaps you can give a filename?
And the given file stores the command(s)?
# 6  
Old 05-13-2018
You might have more success in using awk in filter your control file.

Slight change in format of control file:

Code:
RHEL apple
    echo apple
RHEL bravo 
    ls -l bravo*
RHEL church 
    chmod church.txt

SUSE drive
    chown user1 drive.txt
SUSE eagle 
    echo "eagle flies"
SUSE feather
    ls -l feather*

HP-UX google
    sed 's/^Google.*$/&\
ACTION: go to www.google.com/' google.txt

HP-UX heartache
    ls -l heartache.txt* | awk '{ print $9 }'


I set the variable CGRP to be the 2nd param on your control lines. The awk output could be piped to a shell once your happy it's looking OK:

Code:
OPER_SYS="$1"
  
case $OPER_SYS in
    RHEL|SUSE|HP-UX)
    ;;
    *)  echo "Usage $0 [RHEL|SUSE|HP-UX]" >&2
        exit 2
    ;;
esac

awk -v SYS=$OPER_SYS '
  NF==2 && $1==SYS { exec = 1 ; print "CGRP="$2 ; next }
  NF==2 && $1 ~ "(RHEL|SUSE|HP-UX)" { exec = 0 }
  exec { print }' control


example:
Code:
$ ./do_control HPUX
Usage ./do_control [RHEL|SUSE|HP-UX]

$ ./do_control HP-UX
CGRP=google
    sed 's/^Google.*$/&\
ACTION: go to www.google.com/' google.txt

CGRP=heartache
    ls -l heartache.txt* | awk '{ print $9 }'

This User Gave Thanks to Chubler_XL For This Post:
# 7  
Old 05-14-2018
I apologize to everyone helping out here, I posted such a bad sample of what I'm actually doing. Anyway this one below is much closer to what I'm actually doing.

Code:
RHEL:syslogd:"sed -i 's/^syslogd.*$/&\n*ACTION \/etc\/init.d\/syslog start/' $CFG_DIR/ps_mon.cfg"
RHEL:ntpd:"sed -i 's/^ntpd.*$/&\n*ACTION \/etc\/init.d\/ntpd start/' $CFG_DIR/ps_mon.cfg"
RHEL:scopeux:"sed -i 's/^scopeux.*$/&\n*ACTION \/opt\/perf\/bin\/ovpa start all/' $CFG_DIR/ps_mon.cfg"

SUSE:cron:"sed -i 's/^cron.*$/&\n*ACTION \/etc\/init.d\/cron start/' $CFG_DIR/ps_mon.cfg"
SUSE:scopeux:"sed -i 's/^scopeux.*$/&\n*ACTION \/opt\/perf\/bin\/ovpa start all/' $CFG_DIR/ps_mon.cfg"
SUSE:midaemon:"sed -i 's/^midaemon.*$/&\n*ACTION \/opt\/perf\/bin\/ovpa start all/' $CFG_DIR/ps_mon.cfg"

HP-UX:scopeux:"find $CFG_DIR -name "ps_mon.cfg" | while IFS= read -r file; do sed 's/^scopeux.*$/&@*ACTION \/opt\/perf\/bin\/ovpa start all/' "$file" | tr '@' '\n' > tmp && mv tmp "$file"; done"
HP-UX:midaemon:"find $CFG_DIR -name "ps_mon.cfg" | while IFS= read -r file; do sed 's/^midaemon.*$/&@*ACTION \/opt\/perf\/bin\/ovpa start all/' "$file" | tr '@' '\n' > tmp && mv tmp "$file"; done"
HP-UX:perfalarm:"find $CFG_DIR -name "ps_mon.cfg" | while IFS= read -r file; do sed 's/^perfalarm.*$/&@*ACTION \/opt\/perf\/bin\/ovpa start all/' "$file" | tr '@' '\n' > tmp && mv tmp "$file"; done"

I've just found a way on how to deal with \n special character in HP-UX thus the above. Anyway, I replaced the delimiters with ":" because spaces inside the commands in quotes are problematic. When I try the code below in HP-UX:

Code:
for line in `cat sample.ctl`
do
echo $line
done

I get the output below:

Code:
HP-UX:scopeux:"find
$CFG_DIR
-name
"ps_mon.cfg"
|
while
IFS=
read
-r
file;
do
sed
's/^scopeux.*$/&@*ACTION
\/opt\/perf\/bin\/ovpa
start
all/'
"$file"
|
tr
'@'
'
'
>
tmp
&&
mv
tmp
"$file";
done"

That's why I can't use awk to assign variables. The only way I've seen so far to solve this is to replace the spaces with another character, like a comma, then remove the comma with sed substitution when I assign the command on a variable. Too much of a hassle. If you have a way to simplify this, please let me know.

Thanks for the help so far.

Last edited by The Gamemaster; 05-14-2018 at 02:52 AM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing log file and print latest number in loop

Hello All, I have an awk script which parses my log file and prints number grepping from a specific line/pattern, now i have to come with a shell script to continue reading the log untill the job is completed, which i would know while reading session log untill process encounters a final... (1 Reply)
Discussion started by: Ariean
1 Replies

2. Shell Programming and Scripting

Check and control params in parsing file

Hello, I would like to control and check the right parameters $1 must have 4 alphabetics digits among eora qora pora fora $2 must have 2 numerics digits 00 to 11 $3 must have 2 numerics digits 00 to 59 $4 must have 10 characters alpha numerics as 2013-02-26 For example : In case 5) if i... (15 Replies)
Discussion started by: amazigh42
15 Replies

3. Shell Programming and Scripting

Check and control params in parsing file

Hello, I would like to control and check the right parameters $1 must have 4 alphabetics digits among eora qora pora fora $2 must have 2 numerics digits 00 to 11 $3 must have 2 numerics digits 00 to 59 $4 must have 10 characters alpha numerics as 2013-02-26 For example : In case 5) if i... (1 Reply)
Discussion started by: amazigh42
1 Replies

4. Shell Programming and Scripting

Loop exit control

Hi I would like to exit the loop below on <Enter> even if it sleeps. Is it possible? while true do my_procedure; sleep 60 done Thanks (7 Replies)
Discussion started by: zam
7 Replies

5. UNIX for Dummies Questions & Answers

For loop control with two variables in csh shell

Hi All How can i control for loop with two different variables in csh shell Regards Nikhil (1 Reply)
Discussion started by: Nikhilindurkar
1 Replies

6. Shell Programming and Scripting

Parsing of file for Report Generation (String parsing and splitting)

Hey guys, I have this file generated by me... i want to create some HTML output from it. The problem is that i am really confused about how do I go about reading the file. The file is in the following format: TID1 Name1 ATime=xx AResult=yyy AExpected=yyy BTime=xx BResult=yyy... (8 Replies)
Discussion started by: umar.shaikh
8 Replies

7. Shell Programming and Scripting

Control Not Coming Out Of While Loop

I have an empty .gz file in archival directory. And I am redirecting to a dat file. My while loop is not getting ended. I need the solution. cnt=0 while read line do cnt=`expr $cnt + 1` echo "$ARCH_DIR/$line.gz" >> $DATA_DIR/$FILE_LIST_FILE_FEB FILE_NAMES=${FILE_NAMES}"... (2 Replies)
Discussion started by: vinodh1978
2 Replies

8. Shell Programming and Scripting

for loop control

Hi, I have taken a piece of code from a book, which is working as per the specification. The code.... for entry in * do if then echo $entry fi done The sub-directories present in the current directory will be displayed while executing. ... (3 Replies)
Discussion started by: saravanakumar
3 Replies

9. Shell Programming and Scripting

find command in while loop - how to get control when no files found?

I have the following statement in script: find ${LANDING_FILE_DIR}${BTIME_FILENAME_PATTERN2} -print | while read file; do ... done When there are no files located by the find comand it returns: "find: bad status-- /home/rnitcher/test/....." to the command line How do I get control in... (3 Replies)
Discussion started by: mavsman
3 Replies

10. Shell Programming and Scripting

sending control c in the loop

I want to go through the list of items and run it. while running it, some of them will have either >there is no response # and then end it... so that it can go to next item OR >there is response # but in order to break out of it, u need to do Control c. How do you send control... (6 Replies)
Discussion started by: hankooknara
6 Replies
Login or Register to Ask a Question