Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

More efficent Data Parsing

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting More efficent Data Parsing
# 1  
Old 10-21-2009
More efficent Data Parsing
I am looking for a way to parse out some numbers from text. This is an excerpt from a larger script that I am trying to make run a little smoother.

Specifically this script is used to Capture DV video streams on a linux machine from the terminal. The setup does 6 streams at once, and this part of the script is involved in instructing the particular instance of the script to record from 1 of the 6 DV devices. The DV devices have a GUID that identifys them, and I need to set each device to it's own channel so that streams don't get confused.

This part that I want to clean searches the results from the command plugreport (which I've included below) and finds what HOST ADAPTER and NODE each GUID is currentlty associated with. This information is used later in the script to tell each device what channel to broadcast on (using plugctl, this requires the HOST ADAPTER and NODE for each GUID)

Currently in my script below I run Plugreport 9 times to parse out the 9 possible combinations of HOST ADAPTER and NODE for the 6 GUIDs.

What I would like to do is run Plugreport 1 time and parse all the pertinant numbers to varibles and simply check those all at once...

Here is the mentioned chunk of script

Code:
case 0x$deck_guid in

 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $7}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $3}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $5}')
          XCHANNEL=1
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $11}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $3}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $9}')
          XCHANNEL=2
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $15}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $3}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $3}')
          XCHANNEL=3
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $22}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $18}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $20}')
          XCHANNEL=4
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $26}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $18}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $24}')
          XCHANNEL=5
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $30}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $18}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $28}')
          XCHANNEL=6
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $37}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $33}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $35}')
          XCHANNEL=7
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $41}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $33}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $39}')
          XCHANNEL=8
  ;;
 $(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $45}') )
          HOST_ADAPTER=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $33}')
          NODE=$(plugreport | grep -e Host -e Node |  awk '{printf("%s%s", $0, (NR%20 ? " " : "\n"))}' | awk '{printf $43}')
          XCHANNEL=9
;;
 * ) echo -e "\E[31;40m++ GUID Not Recognized, make sure all 6 ADVC-300's are connected and no other 1394 Devices are connected and try again"
    echo -e "\E[30;47m "
     exit 1
  ;;
esac

And this is what Plugreport outputs (at the time of writing this only 4 of the usual 6 devices are connected, typically there would be 2 more under HOST ADAPTER 2 )
Quote:
Host Adapter 0
==============

Node 0 GUID 0x002011011300557a
------------------------------
oMPR n_plugs=1, data_rate=0, bcast_channel=1
oPCR[0] online=0, bcast_connection=0, n_p2p_connections=2
channel=1, data_rate=0, overhead_id=15, payload=122
iMPR n_plugs=1, data_rate=0
iPCR[0] online=0, bcast_connection=0, n_p2p_connections=0
channel=1

Node 1 GUID 0x0020110113002973
------------------------------
oMPR n_plugs=1, data_rate=0, bcast_channel=2
oPCR[0] online=1, bcast_connection=0, n_p2p_connections=3
channel=2, data_rate=0, overhead_id=0, payload=122
iMPR n_plugs=1, data_rate=0
iPCR[0] online=0, bcast_connection=0, n_p2p_connections=0
channel=2

Node 2 GUID 0x000199000005b5ba
------------------------------
libiec61883 error: error reading oMPR
libiec61883 error: error reading iMPR

Host Adapter 1
==============

Node 0 GUID 0x00201101130008e4
------------------------------
oMPR n_plugs=1, data_rate=0, bcast_channel=4
oPCR[0] online=1, bcast_connection=0, n_p2p_connections=3
channel=4, data_rate=0, overhead_id=15, payload=122
iMPR n_plugs=1, data_rate=0
iPCR[0] online=0, bcast_connection=0, n_p2p_connections=0
channel=4

Node 1 GUID 0x00201101130062a5
------------------------------
oMPR n_plugs=1, data_rate=0, bcast_channel=5
oPCR[0] online=1, bcast_connection=0, n_p2p_connections=2
channel=5, data_rate=0, overhead_id=0, payload=122
iMPR n_plugs=1, data_rate=0
iPCR[0] online=0, bcast_connection=0, n_p2p_connections=0
channel=5

Node 2 GUID 0x0050625600052b2a
------------------------------
libiec61883 error: error reading oMPR
libiec61883 error: error reading iMPR

Host Adapter 2
==============

Node 0 GUID 0x0010dc00015d6b67
------------------------------
libiec61883 error: error reading oMPR
libiec61883 error: error reading iMPR
My first thought was to somehow pipe plugreport to a temporary file and then do my greps and awks from there? If anyone could suggest how I could do this, or perhaps there is a better solution that is much more cool that I just don't know about yet?

Thanks
-Starcast
# 2  
Old 10-21-2009
wouldn't it be easier to run 'plugreport' once, parse it once (for host, node, guid and channel) and do the necessary case/if if needed?
Code:
plugreport | nawk '
  $1 == "Host" { host=$NF; next}
  $1 == "Node" { node=$2; guid=$NF; next}
  /^channel=[0-9][0-9]*$/ {channel=substr($0, index($0, "=")+1); print host, node, guid, channel}' | while read host node guid channel junk
do
   echo "host->[${host} node->[${node}] guid->[${guid}] channel->[${channel}]"
done

# 3  
Old 10-22-2009
Quote:
wouldn't it be easier to run 'plugreport' once, parse it once (for host, node, guid and channel) and do the necessary case/if if needed?
Yes, that's exactly what I'd like to do... just don't know how... and unfortunately my 'nawk' skillz are not quite up to par on this one.

I think your suggestion is actually a bit more complicated than I need. At this point all I need is to find values for $NODE and $HOST_ADAPTER that precede the input variable in the form 0x$deck_guid. No channel input or output needed at this point in the script.

for example if my varible deck_guid=00201101130062a5
then I need to end up with varibles that would be equavalent to NODE=1 and HOST_ADAPTER=1 (Based on the Plugreport quoted earlier)

This is probably part of the code you sent, but as I said I am not really sure what it all means, so I couldn't figure out how to get at the bits I need?

Thanks for your help so far...
-Starcast
# 4  
Old 10-22-2009
Just run script and modify the body of the 'while' to your likings - change 'echo' (once satisfied with the results) to whatever further logic you need.

Something along these lines:
Code:
#!/bin/ksh

myGUID='0x000199000005b5ba'

eval $(plugreport | nawk -v myGUID='0x000199000005b5ba' '
  $1 == "Host" { host=$NF; next}
  $1 == "Node" && $NF==myGUID { node=$2; guid=$NF; printf("NODE=%s; HOST_ADAPTER=%s\n", node, host); exit}')

Last edited by vgersh99; 10-22-2009 at 05:58 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Parsing Bulk Data

Hi All, :D Actullay I am looking for a smart way :b: to parse files in a directory whose count is around 2000000 :eek: in a single day. Find is working with me but taking a lot of times :confused:, sometimes even a day which is not helping me.:wall: So anyone can help me know a smart... (5 Replies)
Discussion started by: jojo123
5 Replies

2. Shell Programming and Scripting

Parsing XML (and insert data) then output data (bash / Solaris)

Hi folks I have a script I wrote that basically parses a bunch of config and xml files works out were to add in the new content then spits out the data into a new file. It all works - apart from the xml and config file format in the new file with XML files the original XML (that ends up in... (2 Replies)
Discussion started by: dfinch
2 Replies

3. Shell Programming and Scripting

Parsing file data

Hey Guys, I'm a novice at shell scripts and i need some help parsing file data. Basically, I want to write a script that retrieves URLs. Here is what I have so far. #!/bin/bash echo "Please enter start date (format: yyyy-mm-dd):\c" read STARTDATE echo "Please enter end date... (7 Replies)
Discussion started by: silverdust
7 Replies

4. Shell Programming and Scripting

Help with parsing data with awk , eliminating unwanted data

Experts , Below is the data: --- Physical volumes --- PV Name /dev/dsk/c1t2d0 VG Name /dev/vg00 PV Status available Allocatable yes VGDA 2 Cur LV 8 PE Size (Mbytes) 8 Total PE 4350 Free PE 2036 Allocated PE 2314 Stale PE 0 IO Timeout (Seconds) default --- Physical volumes ---... (5 Replies)
Discussion started by: rveri
5 Replies

5. Shell Programming and Scripting

Data parsing

Hi, I do have a data file which is divided into compartments by ---------. I would like to extract (parse) some of the data and numbers either using awk or sed The file has the format: CATGC Best GO enrichment: Genes/ORF that have the motifs (genes are sorted by max(pa+pd+po)): ... (6 Replies)
Discussion started by: Lucky Ali
6 Replies

6. Shell Programming and Scripting

Help in Parsing data

I have below string Transaction_ID:SDP-DM-151204679 , Transaction_DateTime:2011-02-11 00:00:15 GMT+05:30 , Transaction_Mode:WAP , Circle_ID:4 , Circle_Name:BJ ,Zone: , CustomerID:B_31563486 , MSISDN:7870904329 , IMSI:405876122068099 , IMEI: , Sub_Profile:Pre-Paid , CPID:Nazara , CPNAME:Nazara ,... (6 Replies)
Discussion started by: poweroflinux
6 Replies

7. Shell Programming and Scripting

Parsing the data

Hi friends, I need to parse the following data in the given format and get the desired output. I need a function, which takes the input as a parameter and the desired output will be returned from the function. INPUT(single parameter as complete string) A;BCF;DFG;FD ... (3 Replies)
Discussion started by: sumesh.1988
3 Replies

8. Shell Programming and Scripting

Parsing data

Hi all , I have a file with billing CDR records in it. I need to parse that information (row format) . The purpose is to compare full content. The example I have given below is a single line record but it has two portions, (1) the line start with “!” and end with “1.2.1.8” and (2) second part... (5 Replies)
Discussion started by: jaygamini
5 Replies

9. Shell Programming and Scripting

Parsing the data

Hi I need to parse the following data using shell script Table ----- stage4n_abc 48 stage4o_abcd 4 adashpg_abc_HeartBeat 1 stage4l_asc 168 Can anyone gimme the solution. I want each value to get stored in an array or variable and want the value to be greped from another file.... (1 Reply)
Discussion started by: Archana.Dheepan
1 Replies

10. Shell Programming and Scripting

Parsing the data in a file

Hi, I have file (FILE.tmp) having contents, FILE.tmp ======== filename=menudata records=0000000000037 ldbname=pinsys timestamp=2005/05/14-18:32:33 I want to parse it bring a new file which will look like, filename records ldbname timestamp... (2 Replies)
Discussion started by: Omkumar
2 Replies
Login or Register to Ask a Question