Help search and replace hex values only in specific files


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Help search and replace hex values only in specific files
Prev   Next
# 1  
Old 06-14-2011
Help search and replace hex values only in specific files

Code:
perl -pi -e 's/\x00/\x0d\x0a/g' `grep -l $'[\x00]GS' filelist` 

This isn't working Smilie, it's not pulling the files that contain the regex. Please help me rewrite this Smilie.

Ideally for this to work on 9K of 20K files in the directory, I've tried this but I don't know enough about awk to make it work.

Code:
for each del in `cat delimiters`
do
  perl -pi -e 's/\x$del/\x0d\x0a/g' `grep -l $'[\x$del]GS' filelist`
done
cat delimiters
15
1c
1e
1f

GOAL
Subsititute the value at position 106 with a newline for the entire file (preferably when the value at pos 107=G) and preferably this will happen on for files that have a value at position 107 and the line begins with ISA.

ISSUE
- my grep statement looking for specific hex values isn't working
- I'd like to identify and perform the same action on files that share the same value at position 106
- some characters are troublesome in a perl one liner or a grep like \ or - or other hex values
- the file contains other non-ascii or non-printable characters throughout the file that should not be substituted
This value at position 106
- changes from file to file
- is consistent within a particular file
- is non-word, usually non-ascii or non-printable (ie hex value 15, 1c, 1e, 1f, 21, 27, 2a, 2e, 3c, 3d, 3e, 3f, 40, 5c, 5e, 60, 7d, 7e, b8, be, c4, 00)
- may not be at position 106 in other files and should not be subsituted in files that do not have this value at pos 106

Code:
INPUT:  This is what the input file looks like.  The value at position 106 is Hex x00 

ISA`00`FTL DATA  `00`FTL DATA  `ZZ`BBBB MFG       `ZZ`FFF  MFG       `110612`1931`U`00401`000001527`0`P`> GS`FA`BBBB MFG`FFF  MFG`110612`1931`940`X`002040 ST`997`000001184 AK1`PO`214 AK2`830`000007903 AK5`A AK9`A`1`1`1 SE`6`000001184 GE`1`940 IEA`1`000001527-
 
or
 
The value at position 106 is Hex x27.  Hex x60 is throughout the file and shouldn't be substituted. 

ISA`00`FTL DATA  `00`FTL DATA  `ZZ`B715 MFG       `ZZ`FTL  MFG       `110612`1931`U`00401`000001527`0`P`>'GS`FA`B715 MFG`FTL  MFG`110612`1931`940`X`002040'ST`997`000001184'AK1`PO`214'AK2`830`000007903'AK5`A'AK9`A`1`1`1'SE`6`000001184'GE`1`940'IEA`1`000001527-
 
OUTPUT:  This is what the output should look like.

ISA`00`FTL DATA  `00`FTL DATA  `ZZ`BBBB MFG       `ZZ`FFF  MFG       `110612`1931`U`00401`000001527`0`P`>
GS`FA`BBBB MFG`FFF  MFG`110612`1931`940`X`002040
ST`997`000001184
AK1`PO`214
AK2`830`000007903
AK5`A
AK9`A`1`1`1
SE`6`000001184
GE`1`940
IEA`1`000001527-

 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Replace hex values using sed command

File lalo.txt contains: Á I need to replace Á by A using sed command. od -x lalo.txt 0000000 c10a 0000002 sed -e 's/\xc1\x0a/A/g' lalo.txt > lalo2.txt Also tried: sed -e 's/\xc3\x81/A/g' lalo.txt > lalo2.txt Output file lalo2.txt still has Á Unix version: SunOS 5.11 ... (9 Replies)
Discussion started by: mrreds
9 Replies

2. Shell Programming and Scripting

Search and replace specific positions of specific lines

Hi, I have a file with hundreds of lines. I want to search for particular lines starting with 4000, search and replace the 137-139 position characters; which will be '000', with '036'. Can all of this be done without opening a temp file and then moving that temp file to the original file name. ... (7 Replies)
Discussion started by: dsid
7 Replies

3. Shell Programming and Scripting

Search Replace Specific Column using RegEx

Have Pipe Delimited File: > BRYAN BAKER|4/4/2015|518 VIRGINIA AVE|TEST > JOE BAXTER|3/30/2015|2233 MockingBird RD|ROW2On 3rd column where the address is located, I want to add a space after every numeric value - basically doing a "s//&\ / ": > BRYAN BAKER|4/4/2015|5 1 8 VIRGINIA AVE|TEST > JOE... (5 Replies)
Discussion started by: svn
5 Replies

4. Shell Programming and Scripting

How to replace with "sed" some hex values by other hex values?

Assume I have a file \usr\home\\somedir\myfile123.txt and I want to replace all occurencies of the two (concatenated) hex values x'AD' x'A0' bytwo other (concatenated) hex values x'20' x'6E' How can I achieve this with the gnu sed tool? Additional question: Is there a way to let sed show... (1 Reply)
Discussion started by: pstein
1 Replies

5. Shell Programming and Scripting

awk to search for specific line and replace nth column

I need to be able to search for a string in the first column and if that string exists than replace the nth column with "-9.99". AW12000012012 2.38 1.51 3.01 1.66 0.90 0.91 1.22 0.82 0.57 1.67 2.31 3.63 0.00 AW12000012013 1.52 0.90 1.20 1.34 1.21 0.67 ... (14 Replies)
Discussion started by: ncwxpanther
14 Replies

6. Shell Programming and Scripting

Query the table and return values to shell script and search result values from another files.

Hi, I need a shell script, which would search the result values from another files. 1)execute " select column1 from table_name" query on the table. 2)Based on the result, need to be grep from .wft files. could please explain about this.Below is the way i am using. #!/bin/sh... (4 Replies)
Discussion started by: Rami Reddy
4 Replies

7. UNIX for Advanced & Expert Users

Search and replace a array values in perl

Hi, i want to search and replace array values by using perl perl -pi -e "s/${d$i]}/${b$j]}" *.xml i am using while loop for the same. if i excute this,it shows "Substitution replacement not terminated at -e line 1.". please tell me what's wrong this line (1 Reply)
Discussion started by: arindam guha
1 Replies

8. Shell Programming and Scripting

Search for multiple string and replace with respective values

Hi, Can anyone help me to search for multiple strings within specified position and replace with respective string value. For example I need to search the string from the position 11 to 20 and if it contain ABC and then replace it by BCDEFGHIJ ... find AABZSDJIK and replace with QWE. and... (4 Replies)
Discussion started by: zooby
4 Replies

9. Programming

searching files for hex or oct values

I have a set of files without extensions. How can I programatically tell if a file is in gzip format? The gzip file format spec RFC 1952 GZIP File Format Specification version 4.3 states that gzip files have certain hex/oct values at the beginning of the file. 1st byte = 0x1f in hex,... (2 Replies)
Discussion started by: daflore
2 Replies

10. Shell Programming and Scripting

highly specific search and replace for a large number of files

hey guys, I have a directory with about 600 files. I need to find a specific word inside a command and replace only that instance of the word in many files. For example, lets say I have a command called 'foo' in many files. One of the input arguments of the 'foo' call is 'bar'. The word 'bar'... (5 Replies)
Discussion started by: ksubrama
5 Replies
Login or Register to Ask a Question