Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Replacing hex characters '\x0D' with '\x0D\x0A'

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Replacing hex characters '\x0D' with '\x0D\x0A'
# 1  
Old 12-03-2009
Replacing hex characters '\x0D' with '\x0D\x0A'
Hello All,

I have a requirement where I need to replaced the hex character - '\x0D' with 2 hex characters - 'x0D' & 'x0A'

I am trying to use SED -
Quote:
sed 's/\x0D/\x0D\x0A/g' < old.file > new.file
But somehow its not working. Any pointers?

Also the hex character '\x0D' can occur anywhere in the line.

Can this also be accomplished through 'tr'
# 2  
Old 12-03-2009
You can use Perl to fix this:

Code:
$
$ # create a file that has \x0D characters at multiple places
$ perl -le 'print "\x0D The quick \x0D brown fox jumps \x0D over the lazy dog.\x0D"' >f0
$
$ # check the file contents
$ cat -vET f0
^M The quick ^M brown fox jumps ^M over the lazy dog.^M$
$
$ # do an inline replace of all \x0D characters to \x0D\x0A
$
$ perl -i.bak -pe 's/(\x0D)/$1\x0A/g' f0
$
$ # now check the file contents again
$ cat -vET f0
^M$
 The quick ^M$
 brown fox jumps ^M$
 over the lazy dog.^M$
$
$

tyler_durden
# 3  
Old 12-03-2009
you forget to escape \ with \.


Code:
sed 's/\\x0D/\\x0D \\x0A/g'

SmilieSmilieSmilieSmilie
This User Gave Thanks to ahmad.diab For This Post:
# 4  
Old 12-03-2009
Quote:
Originally Posted by ahmad.diab
you forget to escape \ with \.


Code:
sed 's/\\x0D/\\x0D \\x0A/g'

SmilieSmilieSmilieSmilie
Doesn't that remove the special meaning of "\x" which states that what follows is a hex code ?

Code:
$
$ printf "abc\x0Ddef" | od -bc
0000000 141 142 143 015 144 145 146
          a   b   c  \r   d   e   f
0000007
$
$ # sed
$ printf "abc\x0Ddef" | sed 's/\\x0D/\\x0D\\x0A/g' | od -bc
0000000 141 142 143 015 144 145 146
          a   b   c  \r   d   e   f
0000007
$
$ # perl
$ printf "abc\x0Ddef" | perl -ne 's/\x0D/\x0D\x0A/g; print' | od -bc
0000000 141 142 143 015 012 144 145 146
          a   b   c  \r  \n   d   e   f
0000010
$

tyler_durden
This User Gave Thanks to durden_tyler For This Post:
# 5  
Old 12-03-2009
Thanks 'durden_tyler' your code worked...but just to mention that it didn't work on Windows...

Same was the case with my 'sed' command I had mentioned at the start. It was not giving the desired results on Windoze but when I executed on Linux it worked like a charm...

I think its due to the fact that interpreters like 'sed', 'Perl' ignores 'CR' (chariage return) characters on windoze..........
# 6  
Old 12-03-2009
Quote:
Originally Posted by paragkalra
Thanks 'durden_tyler' your code worked...but just to mention that it didn't work on Windows...
...
I think its due to the fact that interpreters like 'sed', 'Perl' ignores 'CR' (chariage return) characters on windoze..........
Don't know about "sed" on Windows, but that would be highly unusual for Perl on Windows:

Code:
C:\>
C:\>REM create a file that has \x0D characters at multiple places
C:\>perl -le "print \"\x0D The quick \x0D brown fox jumps \x0D over the lazy dog.\x0D\"" >f0
C:\>
C:\>REM check the file contents
C:\>vis <f0
\015 The quick \015 brown fox jumps \015 over the lazy dog.\015
C:\>
C:\>REM do an inline replace of all \x0D characters to \x0D\x0A
C:\>perl -i.bak -pe "s/(\x0D)/$1\x0A/g" f0
C:\>
C:\>REM now check the file contents again
C:\>vis <f0
\015
 The quick \015
 brown fox jumps \015
 over the lazy dog.\015

C:\>
C:\>REM print file using the DOS "type" command
C:\>type f0
 The quick
 brown fox jumps
 over the lazy dog.

C:\>
C:\>

The perl interpreter used in the example above is from the ActiveState Perl installed using MSI.

"vis" is just a small C program that makes non-printable characters visible in octal code.

tyler_durden
# 7  
Old 12-03-2009
Sed only, and ASCII only?
paragkalra,

Without using Perl or C, may be this is what you were looking for,
a pure sed solution, on a typical Unix environment:

sed 's/\r/\r\n/g' <infile >outfile

At least it works flawlessly on my RH/Fedora... I've tested it using
an input file, as shown by "od -c", like:

0000000 f i r s t l i n e \r s e c o n
0000020 d l i n e \r t h i r d l i n
0000040 e \r
0000042

and having the output file generated like this:

0000000 f i r s t l i n e \r \n s e c o
0000020 n d l i n e \r \n t h i r d l
0000040 i n e \r \n
0000045

Please not that \r and \n are the standard escape sequences
(used by both "od" and "sed") for 0x0D and 0x0A - presuming
we live in an ASCII world, of course. This may not apply if
you are dealing with systems built upon different codings...

Helpful?

Mario.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Losing carriage return (X0D) after running awk command

Hi Forum. I'm running the following awk command to extract the suffix value (pos 38) from the "AM00" record and append to the end of the "AM01" record. awk 'substr($0,13,4)=="AM00" {SUFFIX = substr($0,38,2)} substr($0,13,4)=="AM01" {$0 = $0 SUFFIX} 1' before.txt > after.txt Before.txt:... (2 Replies)
Discussion started by: pchang
2 Replies

2. HP-UX

Replacing Hex Characters In A File Using awk?

Hi guys, First off, i'm a complete noob to UNIX and LINUX so apologies if I don't understand the basics! I have a file which contains a hex value of '0D' at the end of each line when I look at it in a hex viewer. I need to change it so it contains a hex value of '0D0A0A' I thought... (10 Replies)
Discussion started by: AndyBSG
10 Replies

3. Shell Programming and Scripting

sed replacing specific characters and control characters by escaping

sed -e "s// /g" old.txt > new.txt While I do know some control characters need to be escaped, can normal characters also be escaped and still work the same way? Basically I do not know all control characters that have a special meaning, for example, ?, ., % have a meaning and have to be escaped... (11 Replies)
Discussion started by: ijustneeda
11 Replies

4. UNIX for Dummies Questions & Answers

Replacing hex characters

I have the following file consisting of dates and sample measurements: 05��Oct��2010 1.31�� 06��Oct��2010 1.32�� 07��Oct��2010 1.31�� The hex characters are \xc2\xa0 in sequence. I have tried to remove the characters as follows: sed -i '' -e 's/\xc2\xa0//g' file.dat and as follows... (6 Replies)
Discussion started by: figaro
6 Replies

5. Shell Programming and Scripting

Grepping for hex characters - explanation?

Hello, Yesterday I was looking for a way to grep for a tab in the shell, and found this solution in several places: grep $'a' # Grep for the letter 'a' between two tabs I'm fine with most of this, but I don't understand what the $ (dollar sign) before the first quote does. It doesn't work... (7 Replies)
Discussion started by: mregine
7 Replies

6. Shell Programming and Scripting

Convert hex values to displayable characters

Hi, I am a bit stuck with displaying characters. I am having values like below in the proper displayable characters. which I would want to print the actual value on the right hand side. I dont want to create an array because I would have to create 255 different values. isnt there another way of... (17 Replies)
Discussion started by: ahmedwaseem2000
17 Replies

7. Programming

Data formating using C programm with Hex deciamal 'x0d'

:b:Guys, Can some body throw some light on this please..... sprintf(req_line1, "%c%s%c", '\x0b',"TESTING1",'\x0d'); sprintf(req_line2, "%s%c", "TESTING2", '\x0d'); sprintf(req_line3, "%s%c", "Testing3", '\x0d'); sprintf(req_line4, "%s%c%c%c", "Testing4", '\x0d', '\x1c', '\x0d'); ... (6 Replies)
Discussion started by: sudharma
6 Replies

8. UNIX for Dummies Questions & Answers

replacing characters

Hi, I have a script for replacing bad characters in filenames for f in *; do mv $f `echo $f | tr '+' '_'` done; this replaces + for _ But I need to replace all bad characters ? / % + to _ Pls how can i do this in one script ? (3 Replies)
Discussion started by: palmer18
3 Replies

9. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies

10. Shell Programming and Scripting

Replacing all but last Hex characters in a text line

I must remove hex characters 0A and 0D from several fields within an MS Access Table. Since I don't think it can be done in Access, I am trying here. I am exporting a Table from Access (must be fixed length fields, I think, for my idea to work here) into a text format. I then want to run a... (2 Replies)
Discussion started by: BAH
2 Replies
Login or Register to Ask a Question