Shell Programming and Scripting

Hi,

Some xml files that need to be parsed by a script may contain unknown characters in random positions. I can't copy/paste it to show an example because they disappear this way but in a text editor they show up. Below is an example (in the context this character is supposed to be an apostrophe).



Is there a way to identify, and possibly replace, those characters using sed, awk or else?

Thanks for your help

looks more like a problem with charset or locale (Lang ? )to me... on what was produced the xml file?

The xml files are copied from an apple ipod touch.

Found something for you to read:
Character encoding for iOS developers. Or UTF-8 what now? - Matt Galloway

http://www.unicode.org/charts/PDF/U0000.pdf

Thanks. I've read through these and found them quite informative. I'm wondering now how i can use this information somehow.

I suppose i'd need to make some tests and try out different wildcard combinations

Its more a question of knowing Input format (seems UTF ) and needed format on your unix/linux and doing a translation of char codes...

No wildcards needed, look at how UTF8 works, simply remove any characters >=128.

Code:

tr -d '\200-\377' < inputfile > outputfile