Go Back   The UNIX and Linux Forums > Top Forums > Shell Programming and Scripting


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Reply    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 12-05-2012
Registered User
 
Join Date: Jul 2010
Posts: 24
Thanks: 8
Thanked 0 Times in 0 Posts
Problem With Replace Script

Hi,

I posted a topic requesting help with a script to replace certain things in an XML file

http://www.unix.com/shell-programmin...newline-2.html

The replies helped a lot but I found that on big files it didn't work properly.

The file I'm amending is in the following layout

Code:
<FUNCTION>
  <PRODUCTS>
    <PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no">
      <SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER">
        <STOCK_QUANTITY DATA="21"/>
        <STOCK_DATE DATA="1349284740"/>
      </SUPPLIER>
    </PRODUCT>
    <PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no">
      <SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER">
        <STOCK_QUANTITY DATA="21"/>
        <STOCK_DATE DATA="1349284740"/>
      </SUPPLIER>
    </PRODUCT>
    <PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no">
      <SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER">
        <STOCK_QUANTITY DATA="21"/>
        <STOCK_DATE DATA="1349284740"/>
      </SUPPLIER>
    </PRODUCT>
  </PRODUCTS>
</FUNCTION>

I want to convert it to the following format

Code:
<FUNCTION>
<PRODUCTS>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
</PRODUCTS>
</FUNCTION>

Now I can do this on small files using the following

Code:
printf '%s\n' '1,$s/^  *//' '1,$s/  *$//' 'g/^<PRODUCT / .,/^<\/PRODUCT>/j' w | ed -s file

But on large files it finishes and it is in the following format

Code:
<FUNCTION>
<PRODUCTS>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no"><SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER"><STOCK_QUANTITY DATA="21"/><STOCK_DATE DATA="1349284740"/></SUPPLIER></PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no">
<SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER">
<STOCK_QUANTITY DATA="21"/>
<STOCK_DATE DATA="1349284740"/>
</SUPPLIER>
</PRODUCT>
<PRODUCT CODE="PROD1" ACTION="amend" VALIDATE="no">
<SUPPLIER PRODUCT="SUPPPROD1" ACTION="amend" CODE="SUPPLIER">
<STOCK_QUANTITY DATA="21"/>
<STOCK_DATE DATA="1349284740"/>
</SUPPLIER>
</PRODUCT>
</PRODUCTS>
</FUNCTION>

So some of the records are amended but not all.

Could anyone possibly help me to get this to work for the whole file and not just part of it?

Thanks in advance!
Sponsored Links
    #2  
Old 12-05-2012
Registered User
 
Join Date: Sep 2012
Location: Houston, Texas, USA
Posts: 571
Thanks: 0
Thanked 173 Times in 167 Posts
try:

Code:
awk '
/< *\/ *PRODUCT *>/ {ORS="\n";}
/< *PRODUCT / {ORS="";}
{ sub("^ *",""); sub(" *$",""); print $0; }
' input

Sponsored Links
    #3  
Old 12-05-2012
Registered User
 
Join Date: Jul 2010
Posts: 24
Thanks: 8
Thanked 0 Times in 0 Posts
Thanks for the response, but I get the folowing errors

Code:
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 1
awk: illegal statement near line 1

All I was running was

Code:
awk '/< *\/ *PRODUCT *>/ {ORS="\n";}/< *PRODUCT / {ORS="";}{ sub("^ *",""); sub(" *$",""); print $0; }' filename

    #4  
Old 12-05-2012
Yoda's Avatar
Jedi Master
 
Join Date: Jan 2012
Location: Galactic Empire
Posts: 2,310
Thanks: 154
Thanked 738 Times in 710 Posts
If your OS is Solaris or SunOS use nawk instead of awk
Sponsored Links
    #5  
Old 12-05-2012
Registered User
 
Join Date: Sep 2012
Location: Houston, Texas, USA
Posts: 571
Thanks: 0
Thanked 173 Times in 167 Posts
try putting awk script in file a.awk:

Code:
/< *\/ *PRODUCT *>/ {ORS="\n";}
/< *PRODUCT / {ORS="";}
{ sub("^ *",""); sub(" *$",""); print $0; }

the in command line run:

Code:
awk -f a.awk filename

or use nawk as suggested above
Sponsored Links
    #6  
Old 12-06-2012
Registered User
 
Join Date: Jul 2010
Posts: 24
Thanks: 8
Thanked 0 Times in 0 Posts
I appreciate the help, but putting the awk script in a file provides the same error and using nawk doesn't bring back the data I want.

It looks to have stripped most of the data out.

Essentially, I want to remove the leading spaces on each line and then replace

Code:
>
<SUPPLIER

with

Code:
><SUPPLIER


Code:
>
<STOCK

with

Code:
><STOCK


Code:
/>
</SUPPLIER>

with

Code:
/></SUPPLIER>


Code:
>
</PRODUCT>

with

Code:
></PRODUCT>

As I said, using the following code works, but has problems with larger files.

Code:
printf '%s\n' '1,$s/^  *//' '1,$s/  *$//' 'g/^<PRODUCT / .,/^<\/PRODUCT>/j' w | ed -s filename

Is there anything you could suggest?

I can go through each line and manipulate the data but it takes a long time, whereas the printf code works within a minute.


****EDIT****
Oh, I forgot to mention, the OS is Solaris
Sponsored Links
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
VI Search and Replace problem help... kbreitsprecher Homework & Coursework Questions 4 08-16-2011 04:39 AM
problem with replace command with tr repinementer Shell Programming and Scripting 2 08-19-2009 01:00 AM
find and replace problem allrise123 Shell Programming and Scripting 3 06-06-2009 09:47 AM
Multiline replace problem mathis Shell Programming and Scripting 2 02-13-2009 11:46 AM



All times are GMT -4. The time now is 09:00 AM.