Shell Programming and Scripting

Replace with what?

I'm using UTF-8 on the destination (Linux Mint), just like on input. Not sure what charset woud be more appropriate to set before transferring the files. Any suggestion?

Doesn't work, but thanks for the idea

Ideally with the correct corresponding character, but a space or some other default character is also ok

Did you consider the recode and/or iconv tools?

I didn't know about these until you mentionned them. So i gave it a try using all possible charset available like this but none seems to work out. Either it returns the original character or an error

Code:

recode --list|egrep -v '^\/|^:'|while read s; do recode $s < unknown_character.txt 2>/dev/null ; done|egrep ^Unknown

If I remember correctly, you need to specify a source charset AND a target charset to recode. There should be readymade source-target-pairs, though...

Thanks for pointing this out. I've tried different combinations but didn't get much luck. Looks like the original charset wasn't utf-8 after all. Perhaps is it set internally by the app generating the xml files.

Following Corona688's suggestion i think i'm just going to delete the unwanted character using

Code:

 tr -d '\031'

It seems to be the only one that gives trouble (for now) so removing it this way is the easiest solution.

Thanks all for your help