Replace hex values using sed command


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Replace hex values using sed command
# 1  
Old 01-14-2018
Question Replace hex values using sed command

File lalo.txt contains: Á

I need to replace Á by A using sed command.

Code:
od -x lalo.txt

0000000 c10a
0000002

sed -e 's/\xc1\x0a/A/g' lalo.txt > lalo2.txt

Also tried:
Code:
sed -e 's/\xc3\x81/A/g' lalo.txt > lalo2.txt


Output file lalo2.txt still has Á

Unix version: SunOS 5.11 11.3 sun4v sparc sun4v

Any input? Thank you all!
# 2  
Old 01-14-2018
Hi,

I think you might be using the wrong hex value here perhaps. When I do a lookup for the hex value of Á, I get 00C1. Maybe try that as opposed to C10A, and see if it helps ?
# 3  
Old 01-14-2018
Thank you drysdalk!

I tried the suggested.
Code:
sed -e 's/\x00\xc1/A/g' lalo.txt > lalo3.txt
sed -e 's/\xc1\x00/A/g' lalo.txt > lalo3.txt

Still the problem, no replace.
# 4  
Old 01-14-2018
Hi, see if this works for you:
Code:
sed 'y/Á/A/' lalo.txt > lalo2.txt

or
Code:
tr Á A < lalo.txt > lalo2.txt

# 5  
Old 01-14-2018
Thank you Scrutinizer!

For 1st option:
Code:
# sed 'y/Á/A/' lalo.txt > lalo2.txt
sed: command garbled: y/Á/A/

Second, no char changed.

Smilie
# 6  
Old 01-14-2018
Hi,

Hmm, unusual. Can you check one more thing please ? What's the current value of the LANG (and possibly also LC_ALL) environment variable ? if your current locale isn't actually set to a UTF-8 locale, that might explain some of these problems. Either that, or the utilities on the system you're on just can't handle UTF-8 properly at all. I'm having no issues with this at all on my Linux box, so it could always be something Solaris-specific also. If I get the chance I'll try to test it on a SunOS-style box this evening if I can.

---------- Post updated 15-01-18 at 12:07 AM ---------- Previous update was 14-01-18 at 09:42 PM ----------

Hi,

OK, think I might have a SunOS-based solution for you here. Caveat: this was tested on Tribblix, an open-source Illumos based version of Solaris, so it's SunOS-like, but not official "proper" (or paid-for, more to the point) Oracle Solaris. But it should be compatible enough for almost any purpose.

So, the answer I found was to use octal rather than hex. Try this:

Code:
root@tribblix:~/test# cat test.txt
Á
root@tribblix:~/test# file test.txt
test.txt:       data
root@tribblix:~/test# cat test.txt | sed 's/\301/A/g' > test2.txt
root@tribblix:~/test# cat test2.txt
A
root@tribblix:~/test# file test2.txt
test2.txt:      ascii text
root@tribblix:~/test#

It's possible there's a little-endian-versus-big-endian byte order issue with the hex substitution, so rather than try to get my head around that I thought to try to octal, and that seems to do the job. Let us know how you get on.
# 7  
Old 01-15-2018
The line in lalo.txt in post#1 ist a single char followed by a <newline> char. Hex C1 is "extended" ASCII for "Latin capital letter A with acute", which is hex c3 81 in UTF-8.
That's what mayhap garbled your command in post#5 - two bytes for sed's y command.
You need to be very clear about what encoding you use in a) your session, and b) your data files. iconv or recode might do the job for you. Or use octal / hexadecimal representation for sed or tr as drysdalk proposed.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to replace with "sed" some hex values by other hex values?

Assume I have a file \usr\home\\somedir\myfile123.txt and I want to replace all occurencies of the two (concatenated) hex values x'AD' x'A0' bytwo other (concatenated) hex values x'20' x'6E' How can I achieve this with the gnu sed tool? Additional question: Is there a way to let sed show... (1 Reply)
Discussion started by: pstein
1 Replies

2. Shell Programming and Scripting

Help search and replace hex values only in specific files

perl -pi -e 's/\x00/\x0d\x0a/g' `grep -l $'GS' filelist` This isn't working :confused:, it's not pulling the files that contain the regex. Please help me rewrite this :wall:. Ideally for this to work on 9K of 20K files in the directory, I've tried this but I don't know enough about awk... (7 Replies)
Discussion started by: verge
7 Replies

3. Shell Programming and Scripting

sed replace values of column with ordered numbering

Hello, I am trying to write a shell script that will create a gnuplot file. My main problem is that I have a data file with two columns: 1.05929120E+09 5.0000701214792 1.05930096E+09 5.00006386985764 1.05930584E+09 5.00019465404908 1.05931072E+09 5.00031960589719 ... (2 Replies)
Discussion started by: pau
2 Replies

4. Shell Programming and Scripting

Convert hex values to displayable characters

Hi, I am a bit stuck with displaying characters. I am having values like below in the proper displayable characters. which I would want to print the actual value on the right hand side. I dont want to create an array because I would have to create 255 different values. isnt there another way of... (17 Replies)
Discussion started by: ahmedwaseem2000
17 Replies

5. Programming

searching files for hex or oct values

I have a set of files without extensions. How can I programatically tell if a file is in gzip format? The gzip file format spec RFC 1952 GZIP File Format Specification version 4.3 states that gzip files have certain hex/oct values at the beginning of the file. 1st byte = 0x1f in hex,... (2 Replies)
Discussion started by: daflore
2 Replies

6. Shell Programming and Scripting

Replace blank values for string using sed

Hi everyone, I was wondering how could from a file where each row is separated by tabulations, the row values where are in blank replace them by a string or value. My file has this form: 26/01/09 13:45:00 0 0 26/01/09 14:00:00 1495.601318 0 26/01/09 14:15:00 1495.601318 0 ... (4 Replies)
Discussion started by: tonet
4 Replies

7. UNIX for Dummies Questions & Answers

using sed to replace values...

I have a file: $somevar=somevalue $anothervar= $someothervar=45 I'd like to be able to replace the values in the file. I just don't know how exactly to use sed. I was thinking along the lines of: cat file | sed "s/$somevar=/anotherval/g" I was hoping this would make the... (2 Replies)
Discussion started by: mrwatkin
2 Replies

8. Shell Programming and Scripting

Loop with sed command to replace line with sed command in it

Okay, title is kind of confusion, but basically, I have a lot of scripts on a server that I need to replace a ps command, however, the new ps command I'm trying to replace the current one with pipes to sed at one point. So now I am attempting to create another script that replaces that line. ... (1 Reply)
Discussion started by: cbo0485
1 Replies

9. Shell Programming and Scripting

How do i use sed to replace string with values from a dictionary file

I have file 1 with one million rows. one of the fields is "FIRSTNAME" (the string) I have a second file with about 20 first names. JUDE DAVID HOMER CANE ABEL MARTY CARL SONNY STEVE BERT OSCAR MICKY JAMES JOHN GLENN DOUG (3 Replies)
Discussion started by: yoyolandre
3 Replies

10. UNIX for Advanced & Expert Users

Modifying binary file by editing Hex values ?

Hi , i'm using special binary file (lotus notes) and modifying an hexadecimal address range with windows hex editor and it works fine ! The file is an AIX one and i'm forced to transfert (ftp) it before modifying it and re-transfert ! NOW i would do this directly under AIX ! I can... (4 Replies)
Discussion started by: Nicol
4 Replies
Login or Register to Ask a Question