Unix/Linux Go Back    


Shell Programming and Scripting BSD, Linux, and UNIX shell scripting — Post awk, bash, csh, ksh, perl, php, python, sed, sh, shell scripts, and other shell scripting languages questions here.

String parsing help across multiple UNIX platforms

Shell Programming and Scripting


Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 2 Weeks Ago
harleyvrodred harleyvrodred is offline
Registered User
 
Join Date: Jun 2017
Last Activity: 10 June 2017, 3:29 PM EDT
Posts: 6
Thanks: 5
Thanked 0 Times in 0 Posts
String parsing help across multiple UNIX platforms

Need to parse XML like strings from a file.

Using `egrep -A 1 "Panel Temp" "$2" | tail -2` I get the following string:


Code:
<parameter name="Panel Temp" unit="0.1 C"> <value size="1" starttime="06-08-2017 09:36:56.968">95</value>

I want to output:

Code:
{"Panel Temp" 9.5 C}

The 9.5 C is the value of 9 * unit of 0.1 C

I need something that'll work from a shell script on old and new systems. Particularly from IRIX 6.5...Linux 2.4 to Linix 2.6+ kernel based systems.
Korn ksh is the default shell on IRIX and is probably the most limiting factor, i.e. ksh features that'll work on old and new.

Any help appreciated

Moderator's Comments:
String parsing help across multiple UNIX platforms edit by bakunin: please use CODE-tags (for data and terminal output too). Thank you.

Last edited by bakunin; 2 Weeks Ago at 02:56 PM..
Sponsored Links
    #2  
Old Unix and Linux 2 Weeks Ago
bakunin bakunin is offline Forum Staff  
Bughunter Extraordinaire
 
Join Date: May 2005
Last Activity: 22 June 2017, 5:05 AM EDT
Location: In the leftmost byte of /dev/kmem
Posts: 5,446
Thanks: 100
Thanked 1,509 Times in 1,119 Posts
Quote:
Originally Posted by harleyvrodred View Post
Need to parse XML like strings from a file.
As often stated: one can do that, but be aware that in a strict sense you can't do it by regular exprerssions. For instance, in XML it is not guaranteed that tags are on the same line. They could be on different lines, in different order, etc.. Therefore any regexp-based solution short of a real XML-parser will make implicit assumptions about the format of the file which could or could not be correct.

Quote:
Originally Posted by harleyvrodred View Post
I get the following string:


Code:
<parameter name="Panel Temp" unit="0.1 C"> <value size="1" starttime="06-08-2017 09:36:56.968">95</value>

I want to output:

Code:
{"Panel Temp" 9.5 C}

The 9.5 C is the value of 9 * unit of 0.1 C
OK, the part "9 * 0.1" is perhaps a typo and should (i suppose) read "95 * 0.1". I also understand the format of the string you cut out which containsthe relevant tags. But the command producing this string looks fishy. Can you please post a part of your input file (a few lines surrounding the part you want to cut out and parse), because i think this cutting could be done more easily.

Quote:
Originally Posted by harleyvrodred View Post
Korn ksh is the default shell on IRIX and is probably the most limiting factor, i.e. ksh features that'll work on old and new.
As it is i don't think so: ksh comes in two variants, ksh88 and ksh93 and ksh88 is a strict subset of what ksh93 can do. Find out what IRIX uses, but i think writing for ksh88 should put you on the safe side.

Linux, though, is a different matter, depending on what distribution you use. SuSE 11 for instance, had a real AT&T ksh93 when you installed it and called ksh on the command line. SuSE 12, though, dropped the AT&T version from the repository completely and if you invoke ksh you get some "mksh" (which is a not-compatible clone with most ksh-features missing but some bash-features included), but it won't tell you so. I learned that a few weeks ago, when i gave a script i wrote on RHEL (which again has a real ksh when you invoke ksh) to a colleague from the Linux team and we wondered why it didn't work. So be prepared for some surprises.

I hope this helps.

bakunin
Sponsored Links
    #3  
Old Unix and Linux 2 Weeks Ago
drl's Unix or Linux Image
drl drl is offline Forum Advisor  
Registered Voter
 
Join Date: Apr 2007
Last Activity: 24 June 2017, 7:50 AM EDT
Location: Saint Paul, MN USA / BSD, CentOS, Debian, OS X, Solaris
Posts: 2,127
Thanks: 202
Thanked 389 Times in 337 Posts
Hi, harleyvrodred.

Welcome to the forum.
Quote:
Originally Posted by harleyvrodred
...I need something that'll work from a shell script on old and new systems. ...
I'd be inclined to look at perl.

Best wishes ... cheers, drl
    #4  
Old Unix and Linux 2 Weeks Ago
harleyvrodred harleyvrodred is offline
Registered User
 
Join Date: Jun 2017
Last Activity: 10 June 2017, 3:29 PM EDT
Posts: 6
Thanks: 5
Thanked 0 Times in 0 Posts
Example file


Code:
<?xml version="1.0" encoding="UTF-8"?>                                                                                                                          
                                                                                                                                                                
<log cabinet="403541" MLN="9999" USN="0001538397" timeStamp="Mon May 22 21:02:53 2017" timeZone="MST-MDT">                                                  
   <message type="parametricLog" time="Mon May 22 21:02:53 2017" source="Unknown">                                                                              
      <text />                                                                                                                                                  
      <detail>                                                                                                                                                  
         <parametricdata>                                                                                                                                       
            <data>                                                                                                                                              
               <config>                                                                                                                                         
                  <property name="SW Rev">122.33_V01</property>                                                                                          
               </config>                                                                                                                                        
               <group name="Cooling Cabinet">                                                                                                                   
                  <parameter name="Coolant Temp"  unit="0.1 C">                                                                                                 
                     <value size="1"  starttime="06-08-2017 13:40:52.563">293</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:41:42.612">1641</value>                                                                          
                  </parameter>                                                                                                                                  
                  <parameter name="Coolant Temp"  unit="0.1 C">                                                                                                 
                     <value size="1"  starttime="06-08-2017 13:41:52.525">304</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Actual Air Velocity"  unit="fpm">                                                                                            
                     <value size="1"  starttime="06-08-2017 13:41:57.728">877</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:42:42.573">658</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Actual Air Velocity"  unit="fpm">                                                                                            
                     <value size="1"  starttime="06-08-2017 13:42:57.751">869</value>  
                  </parameter>  
                  <parameter name="Cabinet Temp"  unit="0.1 C">                                                                                                                  
                     <value size="1"  starttime="06-08-2017 13:42:59.751">310</value>                                                                                                                                                                                                                                                                                                                              
                  </parameter>                                                                                                                                  
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:43:42.555">1229</value>                                                                          
                  </parameter>                                                                                                                                  
                  <parameter name="Coolant Temp"  unit="0.1 C">                                                                                                 
                     <value size="1"  starttime="06-08-2017 13:44:02.457">293</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:44:42.586">1545</value>                                                                          
                  </parameter>                                                                                                                                  
                  <parameter name="Coolant Temp"  unit="0.1 C">                                                                                                 
                     <value size="1"  starttime="06-08-2017 13:45:02.455">307</value>                                                                           
                  </parameter>                                                                                                                                  
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:45:42.621">819</value>                                                                           
                  </parameter>
                  <parameter name="Cabinet Temp"  unit="0.1 C">                                                                                                                  
                     <value size="1"  starttime="06-08-2017 13:46:29.751">360</value>                                                                                                                                                                                                                                                                                                                              
                  </parameter>                                                                                                                                                                                                                                                                    
                  <parameter name="Electronics PID Control Value"  unit="-">                                                                                    
                     <value size="1"  starttime="06-08-2017 13:46:42.593">1667</value>                                                                          
                  </parameter>                                                                                                                                  
               </group>                                                                                                                                         
            </data>                                                                                                                                             
         </parametricdata>                                                                                                                                      
      </detail>                                                                                                                                                 
   </message>                                                                                                                                                   
</log>

---------- Post updated at 04:23 PM ---------- Previous update was at 04:17 PM ----------

I'm looking to get the latest values for the parameters

Last edited by harleyvrodred; 2 Weeks Ago at 10:00 PM..
Sponsored Links
    #5  
Old Unix and Linux 2 Weeks Ago
RudiC RudiC is online now Forum Staff  
Moderator
 
Join Date: Jul 2012
Last Activity: 24 June 2017, 9:52 AM EDT
Location: Aachen, Germany
Posts: 10,953
Thanks: 277
Thanked 3,367 Times in 3,100 Posts
A bit difficult to look for "Panel Temp" if there's none in the input file.
OK, looking for "Coolant" in lieu, try

Code:
awk '
match ($0, /"Coolant Temp"/)    {printf "{%s", substr ($0, RSTART, RLENGTH)
                                 gsub (/^.*=|"|> *$/, "")
                                 split ($0, UN)
                                 getline
                                 gsub (/<[^>]*>/, "")
                                 print " "UN[1] * $0, UN[2] "}" 
                                }
' file
{"Coolant Temp" 29.3 C}
{"Coolant Temp" 30.4 C}
{"Coolant Temp" 29.3 C}
{"Coolant Temp" 30.7 C}

The Following User Says Thank You to RudiC For This Useful Post:
harleyvrodred (2 Weeks Ago)
Sponsored Links
    #6  
Old Unix and Linux 2 Weeks Ago
harleyvrodred harleyvrodred is offline
Registered User
 
Join Date: Jun 2017
Last Activity: 10 June 2017, 3:29 PM EDT
Posts: 6
Thanks: 5
Thanked 0 Times in 0 Posts
Apologize for the inconsistencies…

This does indeed find every instance and print them out. Nice job!

What I need is for it to find the last instance and return that last one in a string variable so I can pass it onto the next step.

Do you have such an eloquent example of how this might be accomplished as well?

Much appreciated!
Sponsored Links
    #7  
Old Unix and Linux 2 Weeks Ago
Don Cragun's Unix or Linux Image
Don Cragun Don Cragun is offline Forum Staff  
Administrator
 
Join Date: Jul 2012
Last Activity: 24 June 2017, 4:27 AM EDT
Location: San Jose, CA, USA
Posts: 10,396
Thanks: 527
Thanked 3,626 Times in 3,092 Posts
Quote:
Originally Posted by harleyvrodred View Post
Apologize for the inconsistencies…

This does indeed find every instance and print them out. Nice job!

What I need is for it to find the last instance and return that last one in a string variable so I can pass it onto the next step.

Do you have such an eloquent example of how this might be accomplished as well?

Much appreciated!
Without knowing what shell you're using, we can only guess at how the output of a command should be assigned to a shell variable. And, without knowing what operating system you're using, we have to make several other wild assumption that may not work in your environment, but the following seems to do what you seem to what on the MacOS system I'm using when using a Korn shell:

Code:
#!/bin/ksh
variable=$(awk -F'"' -v pattern='Coolant Temp' '
$2 == pattern {
	split($4, t, / /)
	getline
	split($0, m, /[<>]/)
	o = sprintf("{%s %.1f %s}", FS pattern FS, t[1] * m[3], t[2])
}
END {	print o
}' file)

printf 'Last value extracted from file was: %s\n' "$variable"

The command syntax in this script should work with other shells that perform basic command substitutions as required by the POSIX standards (such as bash). If you want to try this on a Solaris/SunOS system, you'll need to change awk to /usr/xpg4/bin/awk or nawk.

The above is a slight variation on the code RudiC suggested that only prints the last occurrence of the data pattern found in the input file,, save the result in a shell variable, and prints the contents of that shell variable before the script exits.

If a file named file contains the sample contents you provided in post #4 in this thread, the output produced by running the above script is:

Code:
Last value extracted from file was: {"Coolant Temp" 30.7 C}


Last edited by Don Cragun; 2 Weeks Ago at 04:34 AM.. Reason: Fix typo pointed out by RudiC: s/FS pat FS/FS pattern FS/
The Following User Says Thank You to Don Cragun For This Useful Post:
harleyvrodred (2 Weeks Ago)
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Parsing OSX UNIX command results which print in multiple lines hungryd Shell Programming and Scripting 7 12-02-2015 12:31 PM
Specific string parsing in Linux/UNIX satishrao Shell Programming and Scripting 1 05-13-2014 12:18 PM
string parsing using UNIX ali123 Shell Programming and Scripting 4 11-09-2011 10:18 AM
Platforms using Unix Megadrink UNIX and Linux Applications 2 11-11-2009 08:53 PM
Parsing of file for Report Generation (String parsing and splitting) umar.shaikh Shell Programming and Scripting 8 03-02-2009 12:38 AM



All times are GMT -4. The time now is 09:54 AM.