Shell Programming and Scripting

Hi Larry,

These control characters that you are seeing are different from most that I have seen before, however if you can make a copy of the file to work on I'll try and help you out here.

In a copy of the file could you try the following for me, I'm actually going to type some of the instructions in without the code tags as they might cause a problem in displaying the correct information.

The first part of the command is as follows;

Code:

:1,$s/

Just as a normal substitution in vi, then you should type in the following key sequence hold down the 'ctrl' key and 'press v' still holding down the 'ctrl' key press 'shift+s'.

This should be followed by the normal end of the statement like.

Code:

/"/g

Could you give that a test and let me know how you get on - on a copy of the file please.

Regards

Dave

Okay, let's see the same line in a different way:

Code:

sed -n l filename.txt | grep "root"

Unfortunately the output from "cat -v" is fiddly to interpret, but this does tell us that these are single bytes.

Just out of interest, what Microsoft program wrote this file?

Managed to generate what I think are these characters:
Note that I don't think that they are ctrl/s and ctrl/t.
Btw: If you type ctrl/V ctrl/S and your terminal hangs, just type ctrl/Q (it's the old xon/xoff stop start code sequence).

Code:

# What they are not:
echo "\0023\0024" | sed -n l
echo "\0023\0024" | cat -v

\023\024$
^S^T

Code:

# What they probably are:
echo "\0223\0224" | sed -n l
echo "\0223\0224" | cat -v

\223\224$
M-^SM-^T

Have delved through some very old notes on Vi and some more recent notes on Vim, seems that you used to be able to do this using the substitute command like this;

Code:

:1,$%s/\\0223/'/g

Or in sed

Code:

sed -e s/\\0223/"/g <oldfile> > newfile

However I'm unable to test this for you, the post that this replaced was incorrect.

Regards

Dave

Sorry gull04, I'm not into the outer fringes of "vi" and if this is a "large" file we may find that "vi" can't cope.
Other readers on this board may be able to advise.

I'm slowly building-up to a unix "tr" command to translate the characters but will need to know what version of "echo" we have so we can generate each character.
The "awk" and "sed" experts will not doubt be poised.

Will need to know:
Can this be sorted out on the Windows platform?
What Operating System and version are you running and what Shell do you use?
How big is the file?
What is the octal code of the funny characters and what character should they be?

Gents,

I am incredibility thankful for all of the approaches you're providing. I'm going to have a quick go with the benefit of your suggestions and will attempt to come back with a consolidated appreciation.

---------- Post updated at 09:31 AM ---------- Previous update was at 09:18 AM ----------

Attempted to run this with the following results:

Code:

[lmoon@oc4562845586 tools]$ sed -e s/\\0223/" /g SP800-53a_HOLD1 > SP800-53a_HOLD1a
> 
> ^C
[lmoon@oc4562845586 tools]$

It put me into an interactive mode (for which I knew not what to enter, so I Ctrl Ced out).

---------- Post updated at 09:32 AM ---------- Previous update was at 09:31 AM ----------

Oh the windows program these where generated with was ultra edit

---------- Post updated at 09:55 AM ---------- Previous update was at 09:32 AM ----------

methyl,

Please find below answers to your questions:

Code:

$ uname -r
2.6.32-220.4.2.el6.x86_64

$ ls -al | grep SP800-53a_HOLD
-rwxrwxrwx.  1 lmoon lmoon   404089 Mar 12 15:51 SP800-53a_HOLD1
-rwxrwxrwx.  1 lmoon lmoon   395269 Jan  5 15:25 SP800-53a_HOLD1.bak

$ echo $BASH_VERSION
4.1.2(1)-release

Interesting that the sed command you sent earlier (that I ^C ed out of) did generate another file with a small delta in size. I did review it and found the target characters to still be there. Still it looks as though something was replaced.

I'm happy to send you the offending document if you're interested. There is no sensitive information in it.

Thanking You,

Larry

---------- Post updated at 09:59 AM ---------- Previous update was at 09:55 AM ----------

Oh,

I'm not able to discern the octal code for those characters. Mind you the ascii table I looked at only had 128 characters with the highest octal code being '177'.

---------- Post updated at 10:04 AM ---------- Previous update was at 09:59 AM ----------

dubdubdub dot asciitable dot com Not allowed to post URLs until I've engaged five posts In good faith, I'll get there.

We can use the unix "tr" command to change the three funny characters you mentioned in post #1:

Code:

# Demo
echo "\0222\0223\0224" | tr '\222\223\224' "'"'""'

'""

Code:

# For example
cat original_file | tr '\222\223\224' "'"'""'  > new_file

Ps. I now realise that 92, 93 and 94 are the hexadecimal interpretation of each character.