We JUST want you to set LC_ALL=C in the environment of the awk, sed, or cut command you're trying to run. We aren't trying to get you to change the environment in your shell for other commands that you might run later. We NEVER suggested that you issue the command:

Code:

LC_ALL=C

by itself.

If cut -c2- is giving you what you want with the locale still set to en_US.UTF-8, cut is acting strangely (or at least inconsistently) when compared to awk with the same input. The cut -c option works on characters (just like awk substr()) and the two byte sequence at the start of each line you have shown us in file1 is NOT a character in any locale that uses UTF-8 as it's underlying codeset.

You could use:

Code:

cut -b3- file1 > file2

to throw away the first two bytes (not characters) of every line from file1 independent of locale.

Or you could use the suggestion RudiC provided:

Code:

LC_ALL=C sed 's/^..//' file1 > file2

or, using the same syntax, with any of the awk suggestions we've provided, such as:

Code:

LC_ALL=c awk '{print subset($0, 3)}' file1 > file2

to set the locale for sed or awk only without affecting the locale that would be used by any other utility you (or anyone else) would run later in that login session or in any other login sessions currently active on your system.