Shell Programming and Scripting

Hi

I have a file which contains wrong XML, There are some garbage characters at the end of line that I want to get rid of. Example:

<request type="product" ><attributes><pair><name>q</name><value><![CDATA[LOL]]></value></pair><pair><name>start</name><value>1</value></pair></attributes></request>�J I�i�Y�Y��'z�3�u�J�5��}���#Q/k;!�ˑ�9Q){_������ŐF
<request type="product"><attributes><pair><name>q</name><value><![CDATA[LOL2]]></value></pair><pair><name>start</name><value>1</value></pair></attributes></request>4/lIT�l��'�c�Oֲ�{�;��_?��(>͏Y�mP��

How can I remove the garbage characters after </request> ? Or in other words, How to remove string between </request> and <request> ?

Please note from <request> to </request> is just one line so

Code:

awk '/<request t/ , /<\/request>/' test.txt

does not work.


My purpose is to extract value when name is "q" (LOL and LOL2) in this case. So if that can be done , easily, I am not bothered about removing the junk characters.


Thank you for your time.

Code:

perl -e ' while(<>){print "$1\n" if (/name>q<\/name><value><(?:!\[CDATA\[)?([^\]]+)\]\]><\/value/);}' test.txt

You Sir, are awesome. 1000 internets to you.

Sorry, my bad.. Didn't read the question completely. Deleted my erroneous solution.

Hi

Just trying to understand your solution, some questions:

1)why did you use "?:" before !\[CDATA

2) What is the reason for putting "(?:!\[CDATA\[)" in parentheses i.e. "(" and ")"

3) What does "?" in the middle do?

4) What does ([^\]]+) do?

Sorry, I am still learning regular expressions. Someday I want to be as good as you. Please help.

I have made the characters in bold for your convenience.

Thank you.

if (/name>q<\/name><value><(?:!\[CDATA\[)?([^\]]+)\]\]><\/value/);

Code:

/name>q<\/name><value>< # literal string
(?: #non capturing parenthesis
!\[CDATA\[)? This block is optional (allows for cases where the data isn't CDATA escaped)
( #begin capture
[^\]]+ # more than one character which isn't a ] (match is greedy so it will capture as many as possible
)#end of capture
(:?\])+ #What I should have said ;) to make the CDATA wrapper genuinely optional
><\/value# string literal
/x # allow comments in regexes so the maintainer doesn't hunt you down and kill you

The contents matched by the capturing parenthesis available then as $1.

Thanks for taking time out and explaining things. $1 is not (?:!\[CDATA\[) even though it is in parenthesis because it is followed by "?"

Or in other words, what is the reason $1 is not set to (?:!\[CDATA\[) even though that is the first expression inside parenthesis?