A script need help


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting A script need help
# 1  
Old 10-02-2019
A script need help

Hi Gurus,
I have below requirement and have no idea how to achieve this.
the input file like below. there are multiple sections in file, each section has multiple lines. I need to find certain lines (value1, value2, value3 are key words for line searching) and generate another file. in some section, maybe some lines missing. for missing line, output "Missing"
My OS is SunOS 5.10 Generic_150400-64 sun4v sparc sun4v
Code:
[section_abc]
xxxxx
xxxx
CITY=ABC
REGION=CDE
STREET=EFG
[section_cde]
xxxxx
xxxx
CITY=xyz
REGION=123
STREET=345
[section_efg]
xxxxx
xxxx
CITY=900
REGION=200
[section_ghi]
xxxxx
xxxx
REGION=500
STREET=600

expected output file like below:

Code:
[section_abc] CITY=ABC REGION=CDE STREET=EFG
[section_cde] CITY=xyz REGION=123 STREET=345
[section_efg] CITY=900 REGION=200 MISSING 
[section_ghi] missing    REGION=500 STREET=600

thanks in advance

Last edited by green_k; 10-02-2019 at 01:07 AM..
# 2  
Old 10-02-2019
Here is one approach using awk:-
Code:
awk -F= '
        /section/ {
                sc = $1
                S[sc]
                next
        }
        /^VALUE/ {
                A[sc FS $1] = $0
        }
        END {
                for ( k in S )
                        print k, A[k FS "VALUE1"] ? A[k FS "VALUE1"] : "MISSING", A[k FS "VALUE2"] ? A[k FS "VALUE2"] : "MISSING", A[k FS "VALUE3"] ? A[k FS "VALUE3"] : "MISSING"
        }
' file

This User Gave Thanks to Yoda For This Post:
# 3  
Old 10-02-2019
And another awk approach:

Code:
awk -v want="VALUE1,VALUE2,VALUE3" -F'[=\\][]' '
function prnsection(i) {
   if(length(section)) {
     printf "%s",section;
     for(i=1;i in keypos;i++) {
       printf " %s", keys[keypos[i]]
       keys[keypos[i]]="MISSING"
     }
     printf "\n"
   }
}
BEGIN {
   for(i=split(want, keypos, ",");i;i--) {
       keys[keypos[i]]="MISSING";
   }
}
NF>2 { prnsection(); section=$0 }
$1 in keys { keys[$1]=$0 };
END { prnsection() }' infile

These 2 Users Gave Thanks to Chubler_XL For This Post:
# 4  
Old 10-02-2019
Quote:
Originally Posted by Yoda
Here is one approach using awk:-
Code:
awk -F= '
        /section/ {
                sc = $1
                S[sc]
                next
        }
        /^VALUE/ {
                A[sc FS $1] = $0
        }
        END {
                for ( k in S )
                        print k, A[k FS "VALUE1"] ? A[k FS "VALUE1"] : "MISSING", A[k FS "VALUE2"] ? A[k FS "VALUE2"] : "MISSING", A[k FS "VALUE3"] ? A[k FS "VALUE3"] : "MISSING"
        }
' file

thanks Yoda for your quick response. the code work with my sample data. it is my bad I didn't provide sample data correctly. the value1 , value2 and value3 don't have any relation. for example: CITY, REGION, STREET. I updated my post.

--- Post updated at 12:08 AM ---

Quote:
Originally Posted by Chubler_XL
And another awk approach:

Code:
awk -v want="VALUE1,VALUE2,VALUE3" -F'[=\\][]' '
function prnsection(i) {
   if(length(section)) {
     printf "%s",section;
     for(i=1;i in keypos;i++) {
       printf " %s", keys[keypos[i]]
       keys[keypos[i]]="MISSING"
     }
     printf "\n"
   }
}
BEGIN {
   for(i=split(want, keypos, ",");i;i--) {
       keys[keypos[i]]="MISSING";
   }
}
NF>2 { prnsection(); section=$0 }
$1 in keys { keys[$1]=$0 };
END { prnsection() }' infile

thanks Chubler_XL for you quick response, the code works as I expected.

Last edited by green_k; 10-02-2019 at 01:15 AM..
# 5  
Old 10-02-2019
Quote:
Originally Posted by Yoda
Here is one approach using awk:-
Code:
awk -F= '
        /section/ {
                sc = $1
                S[sc]
                next
        }
        /^VALUE/ {
                A[sc FS $1] = $0
        }
        END {
                for ( k in S )
                        print k, A[k FS "VALUE1"] ? A[k FS "VALUE1"] : "MISSING", A[k FS "VALUE2"] ? A[k FS "VALUE2"] : "MISSING", A[k FS "VALUE3"] ? A[k FS "VALUE3"] : "MISSING"
        }
' file

Hi Yoda,
I modify this code to match multiple pattern, it works fine. I have below question.
1. what's purpose of FS in this array. A[sc FS $1]. my understanding is when match find, then assign $0 to array A with index "section and $1".
2. is it possible using pattern match at below code value1, value2? in the file, this value are not same. for example:
there are some value like below: in this case, we consider these two city are same.
Code:
[section_abc]
CITY_1=ABC
[section_cde]
CITY_new=xyz

Code:
print k, A[k FS "VALUE1"] ? A[k FS "VALUE1"] : "MISSING", A[k FS "VALUE2"] ? A[k FS "VALUE2"] : "MISSING", A[k FS "VALUE3"] ? A[k FS "VALUE3"] : "MISSING"

# 6  
Old 10-03-2019
Quote:
Originally Posted by Chubler_XL
And another awk approach:

Code:
awk -v want="VALUE1,VALUE2,VALUE3" -F'[=\\][]' '
function prnsection(i) {
   if(length(section)) {
     printf "%s",section;
     for(i=1;i in keypos;i++) {
       printf " %s", keys[keypos[i]]
       keys[keypos[i]]="MISSING"
     }
     printf "\n"
   }
}
BEGIN {
   for(i=split(want, keypos, ",");i;i--) {
       keys[keypos[i]]="MISSING";
   }
}
NF>2 { prnsection(); section=$0 }
$1 in keys { keys[$1]=$0 };
END { prnsection() }' infile

Hi Chubler_XL, thanks for your answer. it works fine. since I am relatively new for unix/awk scripting. I am not able to fully understand the code. below is my understanding about this code, some part I don't know how it works and have some questions. could you please review and give me brief explanation

thanks in advance.

Code:
awk -v want="RECORD_COUNT,VALUE2,VALUE3" -F'[=\\][]' '  --- F'[=\\][]' need to understand how the regular exp works.. 
function prnsection(i) {                                --- function pass arg i in
   if(length(section)) {                                --- if section is not empty do following
     printf "%s",section;                               --- print section
     for(i=1;i in keypos;i++) {                         --- for loop, max i is number of array keypos: keypos[value1]=1, keypos[value2]=2, keypos[values3]=3 
       printf " %s", keys[keypos[i]]                    --- array keys elements are: keys[1]=value1, keys[2]=value2, key[3]=value2 
       keys[keypos[i]]="MISSING"                        --- if array keys element doens't have value , assign value "missing"
     }                                                 
     printf "\n"
   }
}
BEGIN {
   for(i=split(want, keypos, ",");i;i--) {              --- create array keypos element based on variable want
       keys[keypos[i]]="MISSING";                       --- create array keys if keys is empty then assign value missing.
   }
}
NF>2 { prnsection(); section=$0 }                       ---if NF> 2  then call function and assign $0 to section. the function has one 
                                                                                    --- argument, but here is empty, 
														---how the value be passed in?	
														---what's the purpose to call this function?
$1 in keys { keys[$1]=$0 };                             --- first my understanding is $1 is VALUE1, VALUE2..., I tried command, with -F'[=\\][]' 
                                                                           ----as delimiter, NF=1, not sure how it works. 
END { prnsection() }' file                              ---here call the function to print result..

# 7  
Old 10-03-2019
Quote:
Originally Posted by green_k
Hi Chubler_XL, thanks for your answer. it works fine. since I am relatively new for unix/awk scripting. I am not able to fully understand the code. below is my understanding about this code, some part I don't know how it works and have some questions. could you please review and give me brief explanation

thanks in advance.
Glad to explain what is going on in this code. Working thru and understanding is a great way to improve your awk skills.

Field separator RE [=\\][]

This is a simple bracket [] expression and matches any of the following characters as a field separator =, ] and [.
the ] character needs to be escaped in the RE to stop it being interpreted as a close bracket for the list.
We also need to escape the escape to stop the shell eating it up before it's passed to awk.

After the init section the two arrays are populated as follows:
Code:
keypos[1]=RECORD_COUNT
keypos[2]=VALUE2
keypos[3]=VALUE3

keys[RECORD_COUNT]="MISSING"
keys[VALUE2]="MISSING"
keys[VALUE3]="MISSING"

The main use of keypos is to ensure the output is ordered the same as the want list.
If we just iterated thru keys[] the order is arbitrary and may change for different implementations of awk.
In prnsection() we use a for loop starting at i=1 and finishing when i is no longer in keypos ([icode]i in keypos[/code])

They key array is initialized to "MISSING" at the start and at each new section header.

$1 in keys { keys[$1]=$0 };
This code updates the key array when $1 (the part in front of the = sign) is in keys[].

The argument in awk server two purposes 1 is for input purposes the 2nd is to define local variables.
Actual arguments should be specified first followed by any local variables.
Here there are not arguments and i is simply a local variable to prnsection().
Its a good habit to always use local variables in functions unless there is a reason for them to be
global. Imagine if you had a for loop using a counter i and i was not local in prnsection(),
the i would be changed by the function call.
This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

5 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Shell Programming and Scripting

Shell script works fine as a standalone script but not as part of a bigger script

Hello all, I am facing a weird issue while executing a code below - #!/bin/bash cd /wload/baot/home/baotasa0/sandboxes_finance/ext_ukba_bde/pset sh UKBA_publish.sh UKBA 28082015 3 if then echo "Param file conversion for all the areas are completed, please check in your home directory"... (2 Replies)
Discussion started by: ektubbe
2 Replies

3. UNIX for Dummies Questions & Answers

Calling a script from master script to get value from called script

I am trying to call a script(callingscript.sh) from a master script(masterscript.sh) to get string type value from calling script to master script. I have used scripts mentioned below. #masterscript.sh ./callingscript.sh echo $fileExist #callingscript.sh echo "The script is called"... (2 Replies)
Discussion started by: Raj Roy
2 Replies

4. Shell Programming and Scripting

Script will keep checking running status of another script and also restart called script at night

I am using blow script :-- #!/bin/bash FIND=$(ps -elf | grep "snmp_trap.sh" | grep -v grep) #check snmp_trap.sh is running or not if then # echo "process found" exit 0; else echo "process not found" exec /home/Ketan_r /snmp_trap.sh 2>&1 & disown -h ... (1 Reply)
Discussion started by: ketanraut
1 Replies

5. Shell Programming and Scripting

create a shell script that calls another script and and an awk script

Hi guys I have a shell script that executes sql statemets and sends the output to a file.the script takes in parameters executes sql and sends the result to an output file. #!/bin/sh echo " $2 $3 $4 $5 $6 $7 isql -w400 -U$2 -S$5 -P$3 << xxx use $4 go print"**Changes to the table... (0 Replies)
Discussion started by: magikminox
0 Replies
Login or Register to Ask a Question