Shell Programming and Scripting

Hi All,

We have requirement to read XML details from large Log File and save in new file name. I have enclosed the sample xml which has start tag <OUTBOUND_MESSAGE xmlns="http://www.abc.com"> and the end tag as </OUTBOUND_MESSAGE>

The log file contains many lines with other details which should not be considered. There are many occurrence of these XML section and all need to be written in the new file.

When I use following command, I can see the start and end line of the XML which is relevent to me. I find difficulty to read only the lines between these tag. Is there any direct command is there to read the lines between these to lines and save in new file?

ca> egrep -in "<OUTBOUND_MESSAGE|</OUTBOUND_MESSAGE>" Output_20060422131.log
21047:<OUTBOUND_MESSAGE xmlns="http://www.abc.com/SWITCH">
21089:</OUTBOUND_MESSAGE>
22162:<OUTBOUND_MESSAGE xmlns="http://www.abc.com/CELL">
22201:</OUTBOUND_MESSAGE>
22889:<OUTBOUND_MESSAGE xmlns="http://www.abc.com/ANTENNA"
22926:</OUTBOUND_MESSAGE>

<?xml version="1.0" encoding="UTF-8"?>
<OUTBOUND_MESSAGE xmlns="http://www.abc.com">
<HEADER>
<OBJ_IDENT>
<KEY NAME="CMR_CODE">82</KEY>
</OBJ_IDENT>
<METADATA>
<PUBLISHER>MARSS-CMR</PUBLISHER>
<CHANGE_ID>5</CHANGE_ID>
<OPERATION>CREATE</OPERATION>
<CHANGE_TIME>2006-05-03T10:47:50+01:00</CHANGE_TIME>
<PUB_TIME>2006-05-03T11:08:49+01:00</PUB_TIME>
</METADATA>
</HEADER>
<DATA>
<MARSS-CMR>
<cmr_code>82</cmr_code>
<cmr_status>Open</cmr_status>
<cmr_priority>1 - High</cmr_priority>
<cmr_impact>High</cmr_impact>
<cmr_category>Emergency</cmr_category>
<cmr_chndesc>Testing on 03/05/06 (1)</cmr_chndesc>
<cmr_benefdesc>Testing on 03/05/06 (1)</cmr_benefdesc>
<cmr_tsdesc>Testing on 03/05/06 (1)</cmr_tsdesc>
<cmr_planstart>2006-10-01T10:00:00+01:00</cmr_planstart>
<cmr_plancomplete>2006-10-02T10:00:00+01:00</cmr_plancomplete>
</MARSS-CMR>
</DATA>
</OUTBOUND_MESSAGE>

you could use sed to print the pattern in between tags, e.g.

John Daggett, 341 King Road, Plymouth MA
Alice Ford, 22 East Broadway, Richmond VA
Orville Thomas, 11345 Oak Bridge Road, Tulsa OK
Terry Kalkas, 402 Lans Road, Beaver Falls PA
Eric Adams, 20 Post Road, Sudbury MA
Hubert Sims, 328A Brook Road, Roanoke VA
Amy Wilde, 334 Bayshore Pkwy, Mountain View CA
Sal Carpenter, 73 6th Street, Boston MA



this is my infile contents. Now I am only cutting the tage between PA and VA
and putting this segment into the new file say txt.

$sed -n "/PA/,/VA/p" infile > txt1

$cat txt

Terry Kalkas, 402 Lans Road, Beaver Falls PA
Eric Adams, 20 Post Road, Sudbury MA
Hubert Sims, 328A Brook Road, Roanoke VA


hope this helps you out in getting ur things done.

-Manish Jha

Hi,

I am sorry. I don't know sed. But still I tried with this. It is not working. Can you please let me know where I do mistake?

sed -n "/<OUTBOUND*/,/</OUTBOUND>/p" cralog.log > output.log

If you could provide shell solution it will be great other can you please let me know how to fit this sed command to my requirement. Thanks for your reply.

Thinakar

You need to 'escape' the forward-slash in your </OUTBOUND> tag. Use a back-slash to escape it:

No syntax error now. But output is not what as I required. Please see the output..

cnbas-clintgw-1a> sed -n "/<OUTBOUND*/,/<\/OUTBOUND>/p" cramer2router_20060422131927.log | more
<OUTBOUND_MESSAGE>
<HEADER>
<OBJ_IDENT>
<KEY NAME="NODEID">6450200</KEY>
</OBJ_IDENT>
<METADATA>
<PUBLISHER>CRA-SECTOR</PUBLISHER>
<CHANGE_ID>24712</CHANGE_ID>
<OPERATION>UPDATE</OPERATION>
<CHANGE_TIME>2006-03-27T21:06:47+01:00</CHANGE_TIME>
<PUB_TIME>2006-04-22T14:45:45+01:00</PUB_TIME>
<ADD_INFO>MAGNET-PRD-SECTOR-004337G_3</ADD_INFO>
</METADATA>
</HEADER>
<DATA>
<CRA-SECTOR>
<object_type>BTS</object_type>
<nodeid>6450200</nodeid>
<id>MAGNET-PRD-SECTOR-004337G_3</id>
<status>Planned</status>
<install_date>1998-03-28T00:00:00+00:00</install_date>
<external_ref>004337G</external_ref>
<product_id>MAGNET-PRD-PRODUCT-GSM_Cell_Nokia</product_id>
<parent_id>CRAMER-PRD-CELL-004337G</parent_id>
<site_id>MAGNET-PRD-SITE-004337</site_id>
<ATTRIBUTE>
<nodeid>6450200</nodeid>
<attr_name>Cell Id</attr_name>
<attr_value>34337</attr_value>
</ATTRIBUTE>
</CRA-SECTOR>
</DATA>
</OUTBOUND_MESSAGE>
22-Apr-2006 14:45:46 com.logica.eai.adapter.outbound.MOChangePublishingThread publishChangedMO
FINEST: application of activity [2, DB-ADAPTER-OUTPUT-VALIDATE] started...
22-Apr-2006 14:45:46 com.logica.eai.adapter.activities.ValidateMessageActivityFactory$ValidateAdapt
rActivity perform
FINEST: application of validating activity started [DB-ADAPTER-OUTPUT-VALIDATE]...
22-Apr-2006 14:45:46 com.logica.eai.adapter.outbound.MOChangePublishingThread publishChangedMO
FINEST: application of activity [3, null] started...

<?xml version="1.0" encoding="UTF-8"?>
<OUTBOUND_MESSAGE>
<HEADER>
<OBJ_IDENT>
<KEY NAME="NODEID">6422303</KEY>
</OBJ_IDENT>
<METADATA>
<PUBLISHER>CRA-SWITCH</PUBLISHER>
<CHANGE_ID>238</CHANGE_ID>
<OPERATION>UPDATE</OPERATION>
<CHANGE_TIME>2006-03-27T18:31:02+01:00</CHANGE_TIME>
<PUB_TIME>2006-04-22T14:49:36+01:00</PUB_TIME>
<ADD_INFO>CRAMER-PRD-SWITCH-6422303</ADD_INFO>
</METADATA>
</HEADER>
<DATA>
<CRA-SWITCH>
<object_type>MSC</object_type>
<nodeid>6422303</nodeid>
<id>CRAMER-PRD-SWITCH-6422303</id>
<switch_name>MSC29(RUNCORN)</switch_name>
<switch_id>MSC29</switch_id>
<is_active>Y</is_active>
<external_ref>MSC29</external_ref>
<dsi>3</dsi>
<carr_count>0</carr_count>
<chan_tot_count>0</chan_tot_count>
<chan_l1_count>0</chan_l1_count>
<chan_l2_count>0</chan_l2_count>
<chan_l3_count>0</chan_l3_count>
<model>MSC</model>
<manufacturer>Ericsson</manufacturer>
<reporting_area>GSM</reporting_area>
<site_id>MAGNET-PRD-SITE-RN</site_id>
<ATTRIBUTE>
<nodeid>6422303</nodeid>
<attr_name>Element Id</attr_name>
</ATTRIBUTE>
</CRA-SWITCH>
</DATA>
</OUTBOUND_MESSAGE>
22-Apr-2006 14:49:36 com.logica.eai.adapter.outbound.MOChangePublishingThread publishChangedMO
FINEST: application of activity [2, DB-ADAPTER-OUTPUT-VALIDATE] started...
22-Apr-2006 14:49:36 com.logica.eai.adapter.activities.ValidateMessageActivityFactory$ValidateAdapte
rActivity perform
FINEST: application of validating activity started [DB-ADAPTER-OUTPUT-VALIDATE]...
22-Apr-2006 14:49:36 com.logica.eai.adapter.outbound.MOChangePublishingThread publishChangedMO
FINEST: application of activity [3, CDM-TRANSFORM] started...
22-Apr-2006 14:49:36 com.logica.eai.adapter.activities.XSLTActivityFactory$XSLTAdapterActivity perform

After the </OUTBOUND_MESSAGE> tag, it starts print all the lines. But I need only lines between <OUTBOUND_MESSAGE ...> and </OUTBOUND_MESSAGE>. That means, in a large log file, only XML sections tags as I specified above to be printed not additional junk message.

Can you please let me know as soon as possible?

You can use something like this :

Code:

$ cat tags.sh 
while read LINEA
do
   if [[ ! -z $(echo "${LINEA}."|grep "^\<OUTBOUND_MESSAGE") ]]  
   then
      echo "${LINEA}"
      while read LINEA
      do
         echo $LINEA
         if [ "${LINEA}." = '</OUTBOUND_MESSAGE>.' ]
         then
            break
         fi
      done
   fi
done

$ tags.sh <tags.dat #"data file"

Regards

Code:

sed -n -e '/<OUTBOUND_MESSAGE>/{
:top
N
/<\/OUTBOUND_MESSAGE>/b end
b top
:end
s/<OUTBOUND_MESSAGE>\(.*\)<\/OUTBOUND_MESSAGE>/\1/p
}' input.xml