Replacing hex characters '\x0D' with '\x0D\x0A' | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Replacing hex characters '\x0D' with '\x0D\x0A'

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 12-03-2009
paragkalra paragkalra is offline
Registered User
 
Join Date: Nov 2009
Last Activity: 29 December 2010, 8:55 PM EST
Posts: 29
Thanks: 1
Thanked 0 Times in 0 Posts
Replacing hex characters '\x0D' with '\x0D\x0A'

Hello All,

I have a requirement where I need to replaced the hex character - '\x0D' with 2 hex characters - 'x0D' & 'x0A'

I am trying to use SED -
Quote:
sed 's/\x0D/\x0D\x0A/g' < old.file > new.file
But somehow its not working. Any pointers?

Also the hex character '\x0D' can occur anywhere in the line.

Can this also be accomplished through 'tr'
Sponsored Links
    #2  
Old 12-03-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
 
Join Date: Apr 2009
Last Activity: 31 July 2014, 2:21 PM EDT
Posts: 1,840
Thanks: 7
Thanked 265 Times in 240 Posts
You can use Perl to fix this:


Code:
$
$ # create a file that has \x0D characters at multiple places
$ perl -le 'print "\x0D The quick \x0D brown fox jumps \x0D over the lazy dog.\x0D"' >f0
$
$ # check the file contents
$ cat -vET f0
^M The quick ^M brown fox jumps ^M over the lazy dog.^M$
$
$ # do an inline replace of all \x0D characters to \x0D\x0A
$
$ perl -i.bak -pe 's/(\x0D)/$1\x0A/g' f0
$
$ # now check the file contents again
$ cat -vET f0
^M$
 The quick ^M$
 brown fox jumps ^M$
 over the lazy dog.^M$
$
$

tyler_durden
Sponsored Links
    #3  
Old 12-03-2009
ahmad.diab's Avatar
ahmad.diab ahmad.diab is offline
Registered User
 
Join Date: May 2008
Last Activity: 26 November 2013, 11:00 AM EST
Location: Amman Jordan in MEA
Posts: 645
Thanks: 2
Thanked 19 Times in 19 Posts
you forget to escape \ with \.



Code:
sed 's/\\x0D/\\x0D \\x0A/g'

The Following User Says Thank You to ahmad.diab For This Useful Post:
Bill L. (10-27-2011)
    #4  
Old 12-03-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
 
Join Date: Apr 2009
Last Activity: 31 July 2014, 2:21 PM EDT
Posts: 1,840
Thanks: 7
Thanked 265 Times in 240 Posts
Quote:
Originally Posted by ahmad.diab View Post
you forget to escape \ with \.



Code:
sed 's/\\x0D/\\x0D \\x0A/g'

Doesn't that remove the special meaning of "\x" which states that what follows is a hex code ?


Code:
$
$ printf "abc\x0Ddef" | od -bc
0000000 141 142 143 015 144 145 146
          a   b   c  \r   d   e   f
0000007
$
$ # sed
$ printf "abc\x0Ddef" | sed 's/\\x0D/\\x0D\\x0A/g' | od -bc
0000000 141 142 143 015 144 145 146
          a   b   c  \r   d   e   f
0000007
$
$ # perl
$ printf "abc\x0Ddef" | perl -ne 's/\x0D/\x0D\x0A/g; print' | od -bc
0000000 141 142 143 015 012 144 145 146
          a   b   c  \r  \n   d   e   f
0000010
$

tyler_durden
The Following User Says Thank You to durden_tyler For This Useful Post:
Bill L. (05-10-2012)
Sponsored Links
    #5  
Old 12-03-2009
paragkalra paragkalra is offline
Registered User
 
Join Date: Nov 2009
Last Activity: 29 December 2010, 8:55 PM EST
Posts: 29
Thanks: 1
Thanked 0 Times in 0 Posts
Thanks 'durden_tyler' your code worked...but just to mention that it didn't work on Windows...

Same was the case with my 'sed' command I had mentioned at the start. It was not giving the desired results on Windoze but when I executed on Linux it worked like a charm...

I think its due to the fact that interpreters like 'sed', 'Perl' ignores 'CR' (chariage return) characters on windoze..........
Sponsored Links
    #6  
Old 12-03-2009
durden_tyler's Avatar
durden_tyler durden_tyler is offline Forum Advisor  
Registered User
 
Join Date: Apr 2009
Last Activity: 31 July 2014, 2:21 PM EDT
Posts: 1,840
Thanks: 7
Thanked 265 Times in 240 Posts
Quote:
Originally Posted by paragkalra View Post
Thanks 'durden_tyler' your code worked...but just to mention that it didn't work on Windows...
...
I think its due to the fact that interpreters like 'sed', 'Perl' ignores 'CR' (chariage return) characters on windoze..........
Don't know about "sed" on Windows, but that would be highly unusual for Perl on Windows:


Code:
C:\>
C:\>REM create a file that has \x0D characters at multiple places
C:\>perl -le "print \"\x0D The quick \x0D brown fox jumps \x0D over the lazy dog.\x0D\"" >f0
C:\>
C:\>REM check the file contents
C:\>vis <f0
\015 The quick \015 brown fox jumps \015 over the lazy dog.\015
C:\>
C:\>REM do an inline replace of all \x0D characters to \x0D\x0A
C:\>perl -i.bak -pe "s/(\x0D)/$1\x0A/g" f0
C:\>
C:\>REM now check the file contents again
C:\>vis <f0
\015
 The quick \015
 brown fox jumps \015
 over the lazy dog.\015

C:\>
C:\>REM print file using the DOS "type" command
C:\>type f0
 The quick
 brown fox jumps
 over the lazy dog.

C:\>
C:\>

The perl interpreter used in the example above is from the ActiveState Perl installed using MSI.

"vis" is just a small C program that makes non-printable characters visible in octal code.

tyler_durden
Sponsored Links
    #7  
Old 12-03-2009
Mario Impronta Mario Impronta is offline
Registered User
 
Join Date: Dec 2009
Last Activity: 19 April 2010, 4:00 PM EDT
Location: São Paulo, Brazil
Posts: 3
Thanks: 0
Thanked 0 Times in 0 Posts
Sed only, and ASCII only?

paragkalra,

Without using Perl or C, may be this is what you were looking for,
a pure sed solution, on a typical Unix environment:

sed 's/\r/\r\n/g' <infile >outfile

At least it works flawlessly on my RH/Fedora... I've tested it using
an input file, as shown by "od -c", like:

0000000 f i r s t l i n e \r s e c o n
0000020 d l i n e \r t h i r d l i n
0000040 e \r
0000042

and having the output file generated like this:

0000000 f i r s t l i n e \r \n s e c o
0000020 n d l i n e \r \n t h i r d l
0000040 i n e \r \n
0000045

Please not that \r and \n are the standard escape sequences
(used by both "od" and "sed") for 0x0D and 0x0A - presuming
we live in an ASCII world, of course. This may not apply if
you are dealing with systems built upon different codings...

Helpful?

Mario.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
replacing characters with awk thoreman Shell Programming and Scripting 2 03-16-2009 12:38 PM
Replacing Characters with | ooilinlove Shell Programming and Scripting 4 09-12-2008 06:44 AM
Replacing Characters karansachdeva Shell Programming and Scripting 11 09-12-2008 04:46 AM
replacing characters infyanurag Shell Programming and Scripting 3 05-21-2008 01:55 AM
replacing characters palmer18 UNIX for Dummies Questions & Answers 3 08-20-2007 09:58 AM



All times are GMT -4. The time now is 07:40 PM.