Replacing Hex Characters In A File Using awk?


 
Thread Tools Search this Thread
Operating Systems HP-UX Replacing Hex Characters In A File Using awk?
# 8  
Old 04-28-2014
OK, it appears I have muddied the waters by only explaining what I think I need.

Given that my knowledge is poor at best i'll explain the whole problem to try and give it context which will hopefully help.

I have a file which contains a hex value of 01 at the beginning of each 'record' and a hex value of '03' at the end of each record.

Within each record there are several 'lines' of data that end with a hex value of '0D0A'

So the data would look like this (crlf signifies a carriage return/line feed)

Code:
^AHIcrlf
BYEcrlf
-}^C

If I look at it in a HEX viewer I see

Code:
01 48 49 0D 0A
42 59 45 0D 0A
2D 7D 03

The problem I have is that the application I need to put the file into has the following restrictions

- The ^A character(hex value of 01) needs to be replaced with a space(hex value 20). So, I want to replace hex 01 with hex 20

- The ^C character(hex value 03) needs to be replaced with a carriage return and two line feeds(hex values 0D 0A 0A). So, I want to replace hex 03 with hex 0D 0A 0A.

What I want to end up with is

Code:
20 48 49 0D 0A
42 59 45 0D 0A
2D 7D 0D 0A 0A

Hopefully that makes what i'm aiming for a little clearer and apologies for not being accurate enough initially.
# 9  
Old 04-28-2014
Looks like your file is form DOS/win ... Try first using dos2ux then tell us what is left that needs modification...
# 10  
Old 04-28-2014
Try:
Code:
awk '{gsub(/\001/,FS); gsub(/\003/,"\r\n\n")}1' file > newfile


Last edited by Scrutinizer; 04-28-2014 at 01:42 PM.. Reason: Added quote
# 11  
Old 04-29-2014
Quote:
Originally Posted by Scrutinizer
Try:
Code:
awk '{gsub(/\001/,FS); gsub(/\003/,"\r\n\n")}1' file > newfile

That worked like a charm thank you!
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replacing characters in a file

Suppose I have a file which has 1000 columns (5 SHOWN FOR EXAMPLE) two alphabets are separated by a space and then tab A A"\t"C C"\t"G G"\t"0 0"\t"T T A G"\t"C C"\t"G G"\t"A T"\t"0 0 G A"\t"0 0"\t"G C"\t"A A"\t"T C whenever there is a 0 0 in any column, the output should be printed as A... (12 Replies)
Discussion started by: rossi
12 Replies

2. UNIX for Dummies Questions & Answers

Replacing hex characters

I have the following file consisting of dates and sample measurements: 05��Oct��2010 1.31�� 06��Oct��2010 1.32�� 07��Oct��2010 1.31�� The hex characters are \xc2\xa0 in sequence. I have tried to remove the characters as follows: sed -i '' -e 's/\xc2\xa0//g' file.dat and as follows... (6 Replies)
Discussion started by: figaro
6 Replies

3. Shell Programming and Scripting

Replacing hex characters '\x0D' with '\x0D\x0A'

Hello All, I have a requirement where I need to replaced the hex character - '\x0D' with 2 hex characters - 'x0D' & 'x0A' I am trying to use SED - But somehow its not working. Any pointers? Also the hex character '\x0D' can occur anywhere in the line. Can this also be accomplished... (6 Replies)
Discussion started by: paragkalra
6 Replies

4. Shell Programming and Scripting

replacing characters with awk

Hi there! I'm new in this and probably is a quite simple task but I still cannot manage to do it: I have to read some files line by line and then change the input format into another one, but the very first step is to replace the empty variables by error values. I mean, each line looks like:... (2 Replies)
Discussion started by: thoreman
2 Replies

5. UNIX for Dummies Questions & Answers

Replacing characters in csv file

Hello all, This is my first post here, so please excuse me if this question is too obvious or has been asked before. I am new to Unix and although I tried to search your forum for the answer to my question, I could not find an answer that would help me. I have a 500MB csv file with numeric values... (1 Reply)
Discussion started by: finwhiz
1 Replies

6. UNIX for Dummies Questions & Answers

replacing the characters in a file

hi i want to replace the characters between positions 2 to 30 in each line in a file how to do it suggestions welcome (2 Replies)
Discussion started by: trichyselva
2 Replies

7. Shell Programming and Scripting

Help Replacing Characters in Flat File

I was wondering if somebody could help me with something on UNIX. I have a file that looks like this - "nelson,bill","bill","123 Main St","Mpls","MN",55444,8877,william I want to replace all comma with pipes (|), except if the comma is within double quotes. (The first field is an example of... (8 Replies)
Discussion started by: nelson553011
8 Replies

8. UNIX for Dummies Questions & Answers

replacing few characters in a file

Hi All, I have huge xml file. The file contains some comment tags . I have requirement to replace comment tag with another comment tag. Say for example : file X has -- Part of the file <?xml version="1.0" encoding="ISO-2"?><translationResults jobDate="20070123 23:20:51"... (1 Reply)
Discussion started by: purnakarthik
1 Replies

9. HP-UX

Hex characters of ascii file

Hi, Whats the command or how do you display the hexadecimal characters of an ascii file. thanks Bud (2 Replies)
Discussion started by: budrito
2 Replies

10. Shell Programming and Scripting

Replacing all but last Hex characters in a text line

I must remove hex characters 0A and 0D from several fields within an MS Access Table. Since I don't think it can be done in Access, I am trying here. I am exporting a Table from Access (must be fixed length fields, I think, for my idea to work here) into a text format. I then want to run a... (2 Replies)
Discussion started by: BAH
2 Replies
Login or Register to Ask a Question