Replacing Hex Characters In A File Using awk?

Login or Register to Ask a Question and Join Our Community

Replacing Hex Characters In A File Using awk?

Tags

Login to Discuss or Reply to this Discussion in Our Community

Operating Systems HP-UX Replacing Hex Characters In A File Using awk?

04-28-2014

Registered User

5, 0

Join Date: Apr 2014

Last Activity: 30 April 2014, 4:09 AM EDT

Posts: 5

Thanks Given: 0

Thanked 0 Times in 0 Posts

Replacing Hex Characters In A File Using awk?

Hi guys,

First off, i'm a complete noob to UNIX and LINUX so apologies if I don't understand the basics!

I have a file which contains a hex value of '0D' at the end of each line when I look at it in a hex viewer.

I need to change it so it contains a hex value of '0D0A0A'

I thought the easiest way would be using awk so I tried this

Code:

awk '{sub("\x0D","\x0D\x0A\x0A")}' filename

It doesn't give me an error but neither does it seem to work. Can anyone tell me what i'm doing wrong?

Thanks for any advice!

Last edited by AndyBSG; 04-28-2014 at 11:46 AM..

AndyBSG

View Public Profile for AndyBSG

Find all posts by AndyBSG

04-28-2014

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

Could you post a sample of the file?

Last edited by Scrutinizer; 04-28-2014 at 01:37 PM..

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

04-28-2014

Registered User

237, 67

Join Date: Apr 2014

Last Activity: 25 June 2015, 2:11 AM EDT

Location: I for India

Posts: 237

Thanks Given: 18

Thanked 67 Times in 67 Posts

try

Code:

cat filename| sed 's/0D/0D0A0A/g' > file2 && mv file2 filename

Makarand Dodmis

View Public Profile for Makarand Dodmis

Find all posts by Makarand Dodmis

04-28-2014

Registered User

5, 0

Join Date: Apr 2014

Last Activity: 30 April 2014, 4:09 AM EDT

Posts: 5

Thanks Given: 0

Thanked 0 Times in 0 Posts

Here is an example of the file in question

Code:

 
^A{1:F01BNPAGB20AXXX0000000000}{2:I103BARCGB20XXXXN}{3:{108:ORLD190077833XXX}}{4:^M
:20:CCT44195^M
:23B:CRED^M
:32A:140428GBP627,95^M
:33B:GBP632,95^M
:50K:/CH5708686001085122004/TR32793333^M
MSC CROCIERE SA^M
CHEMIN RIEU 12-14^M
GENEVE//CH^M
:52A:BPPBCHGGXXX^M
:57A:BARCGB20XXX^M
:59:/GB65BARC20789880480169^M
Gorkana Group Ltd^M
Discovery House,  28-42 EC1Y8QE Lon^M
don//GB^M
:70:MSC CROCIERE SA - Gorkana Group Ltd^M
. RIF. 647413 30/11/2013^M
:71A:SHA^M
:71F:GBP5,^M
-}^C                                                         ^A{1:F01BNPAGB20AXXX0000000000}{2:I103BARCGB20XXXXN}{3:{108:ORLD190077835XXX}}{4:^M
:20:CCT44196^M
:23B:CRED^M
:32A:140428GBP2703,88^M
:33B:GBP2708,88^M
:50K:/CH5708686001085122004/TR32793355^M
MSC CROCIERE SA^M
CHEMIN RIEU 12-14^M
GENEVE//CH^M
:52A:BPPBCHGGXXX^M
:57A:NWBKGB2LXXX^M
:59:/GB94NWBK50410145129827^M
Worldchoice Enterprises Ltd^M
Worldchoice House PE2 6FT PETERBORO^M
UGH//GB^M
:70:MSC CROCIERE SA - Worldchoice Enter^M
prises Ltd. RIF. 8375 21/10/2013^M
:71A:SHA^M
:71F:GBP5,^M
-}^C                                     ^A

Last edited by AndyBSG; 04-28-2014 at 11:47 AM..

AndyBSG

View Public Profile for AndyBSG

Find all posts by AndyBSG

04-28-2014

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

Try:

Code:

tr -d '\001' < file > newfile

To remove it. In the first post you want to change it to something, but I am not sure to what..

Could you run a relevant part go your file through od -c ?

Last edited by Scrutinizer; 04-28-2014 at 11:51 AM..

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

04-28-2014

Registered User

5, 0

Join Date: Apr 2014

Last Activity: 30 April 2014, 4:09 AM EDT

Posts: 5

Thanks Given: 0

Thanked 0 Times in 0 Posts

I am trying to change a hex value of 0D to 0D0D0A so I really need to replace one character with three characters which is what I think is making it tricky.

AndyBSG

View Public Profile for AndyBSG

Find all posts by AndyBSG

04-28-2014

Moderator

12,296, 3,792

Join Date: Nov 2008

Last Activity: 1 January 2021, 1:47 AM EST

Location: Amsterdam

Posts: 12,296

Thanks Given: 679

Thanked 3,792 Times in 3,282 Posts

It seems to me that there lines in DOS format, that are ending with CR (0D) - LF (0A) , and there are also FF characters (0C). But what do you want to do with them? Why do you want to change 0D to 0D0D0A ?

Scrutinizer

View Public Profile for Scrutinizer

Find all posts by Scrutinizer

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replacing characters in a file

Suppose I have a file which has 1000 columns (5 SHOWN FOR EXAMPLE) two alphabets are separated by a space and then tab A A"\t"C C"\t"G G"\t"0 0"\t"T T A G"\t"C C"\t"G G"\t"A T"\t"0 0 G A"\t"0 0"\t"G C"\t"A A"\t"T C whenever there is a 0 0 in any column, the output should be printed as A...

2. UNIX for Dummies Questions & Answers

Replacing hex characters

I have the following file consisting of dates and sample measurements: 05��Oct��2010 1.31�� 06��Oct��2010 1.32�� 07��Oct��2010 1.31�� The hex characters are \xc2\xa0 in sequence. I have tried to remove the characters as follows: sed -i '' -e 's/\xc2\xa0//g' file.dat and as follows...

3. Shell Programming and Scripting

Replacing hex characters '\x0D' with '\x0D\x0A'

Hello All, I have a requirement where I need to replaced the hex character - '\x0D' with 2 hex characters - 'x0D' & 'x0A' I am trying to use SED - But somehow its not working. Any pointers? Also the hex character '\x0D' can occur anywhere in the line. Can this also be accomplished...

4. Shell Programming and Scripting

replacing characters with awk

Hi there! I'm new in this and probably is a quite simple task but I still cannot manage to do it: I have to read some files line by line and then change the input format into another one, but the very first step is to replace the empty variables by error values. I mean, each line looks like:...

5. UNIX for Dummies Questions & Answers

Replacing characters in csv file

Hello all, This is my first post here, so please excuse me if this question is too obvious or has been asked before. I am new to Unix and although I tried to search your forum for the answer to my question, I could not find an answer that would help me. I have a 500MB csv file with numeric values...

6. UNIX for Dummies Questions & Answers

replacing the characters in a file

hi i want to replace the characters between positions 2 to 30 in each line in a file how to do it suggestions welcome

7. Shell Programming and Scripting

Help Replacing Characters in Flat File

I was wondering if somebody could help me with something on UNIX. I have a file that looks like this - "nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william I want to replace all comma with pipes (|), except if the comma is within double quotes. (The first field is an example of...

8. UNIX for Dummies Questions & Answers

replacing few characters in a file

Hi All, I have huge xml file. The file contains some comment tags . I have requirement to replace comment tag with another comment tag. Say for example : file X has -- Part of the file <?xml version="1.0" encoding="ISO-2"?><translationResults jobDate="20070123 23:20:51"...

9. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud

10. Shell Programming and Scripting

Replacing all but last Hex characters in a text line

I must remove hex characters 0A and 0D from several fields within an MS Access Table. Since I don't think it can be done in Access, I am trying here. I am exporting a Table from Access (must be fixed length fields, I think, for my idea to work here) into a text format. I then want to run a...

Login or Register to Ask a Question