how to get data from hex file using SED or AWK based on pattern sign Post: 302563723

Sponsored Content

Top Forums Shell Programming and Scripting how to get data from hex file using SED or AWK based on pattern sign Post 302563723 by sameucho on Wednesday 12th of October 2011 04:10:03 AM

10-12-2011

Registered User

Hi Alister,

it is a binary file. The Hexdump output of the file is the following:

Code:


00000000  10 00 00 00 5a 01 00 00  00 00 a8 61 00 00 00 00  |....Z......a....|
00000010  00 02 fc 00 dd 01 01 00  01 00 d7 b4 81 d4 80 01  |................|
00000020  12 83 08 32 01 01 00 00  00 00 f1 84 08 21 43 65  |...2.........!Ce|
00000030  87 09 21 13 f0 a5 06 80  04 87 f7 4a 37 86 03 00  |..!........J7...|
00000040  01 0b 87 01 02 88 02 00  01 89 02 00 01 8a 04 03  |................|
00000050  00 00 00 ab 06 80 04 4a  35 05 01 8c 0c 77 77 77  |.......J5....www|
00000060  2e 74 65 73 74 30 2e 64  65 8d 02 f1 21 ae 08 a0  |.test0.de...!...|
00000070  06 80 04 00 00 00 02 af  36 30 34 81 0d 00 00 00  |........604.....|
00000080  00 00 00 00 00 00 00 00  00 00 82 0d 02 1d 82 1f  |................|
00000090  72 96 87 87 43 fb ff ff  00 83 02 28 00 84 02 28  |r...C......(...(|
000000a0  00 85 01 02 86 09 11 10  11 15 55 45 2b 02 00 90  |..........UE+...|
000000b0  09 11 10 11 15 55 41 2b  02 00 91 01 04 93 01 00  |.....UA+........|
....
....

In fact this is ASN1 encoded record (starting in this case at ofset 0x1B) which looks in external decoder like this:

Code:


B4 81 D4 (212d)
.  80 01 (1d): 12 
.  83 08 (8d): 32 01 01 00 00 00 00 F1 
.  84 08 (8d): 21 43 65 87 09 21 13 F0 
.  A5 06 (6d)
.  80 04 (4d): 87 F7 4A 37 
.  86 03 (3d): 00 01 0B 
.  87 01 (1d): 02 
.  88 02 (2d): 00 01 
.  89 02 (2d): 00 01 
.  8A 04 (4d): 02 00 00 00 
.  AB 06 (6d)
.  80 04 (4d): 4A 35 05 01 
.  8C 0C (12d): 77 77 77 2E 74 65 73 74 30 2E 64 65 
.  8D 02 (2d): F1 21 
.  AE 08 (8d)
.  A0 06 (6d)
.  80 04 (4d): 00 00 00 01 
.  AF 36 (54d)
.     30 34 (52d)
.        81 0D (13d): 00 00 00 00 00 00 00 00 00 00 00 00 00 
.        82 0D (13d): 02 1D 82 1F 72 96 87 87 43 FB FF FF 00 
.        83 02 (2d): 28 00 
.        84 02 (2d): 28 00 
.        85 01 (1d): 02 
.        86 09 (9d): 11 10 11 15 54 18 2B 02 00 
.  90 09 (9d): 11 10 11 15 54 14 2B 02 00 
.  91 01 (1d): 04 
.  93 01 (1d): 00 
...
...

I need to get values under TAGs [B4 [80 01 12] ... [AF...[83] X [84] Y]]
where B4 [80 01 12] is the identifier of correct [AF] TAG. To have it universal the full ASN1 decoder would have to be written, but because I need to get just those two values I simplified the task supposing there will not be another occurence of such combination of 'sign' bytes before the desired values which are to be collected.

Last edited by sameucho; 10-12-2011 at 05:36 AM.. Reason: Editorial changes for better readability

sameucho

View Public Profile for sameucho

Find all posts by sameucho

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file based on pattern in awk, grep, sed or perl

Hi All, Can someone please help me write a script for the following requirement in awk, grep, sed or perl. Buuuu xxx bbb Kmmmm rrr ssss uuuu Kwwww zzzz ccc Roooowwww eeee Bxxxx jjjj dddd Kuuuu eeeee nnnn Rpppp cccc vvvv cccc Rhhhhhhyyyy tttt Lhhhh rrrrrssssss Bffff mmmm iiiii Ktttt...

2. Shell Programming and Scripting

Truncating FILE data BASED ON A PATTERN

HI I HAVE A PROBLEM,MY SOURCE FILE IS OF PATTERN S1,E-Certified,29,29,2.7,Certified,4,3,2.7,,0,0,0 S2,Certified,4,3,2.7,,0,0,0,,0 S3,E-Certified,29,29,2.7,,0,0,0 S4,,0,0,0,,0,0,0,,0,0,0,,0,0,0 AND THE EXPECTED OUTPUT IS S1,E-Certified,29,29,2.7 S1,Certified,4,3,2.7...

3. Shell Programming and Scripting

Merge two file data together based on specific pattern match

My input: File_1: 2000_t g1110.b1 abb.1 2001_t g1111.b1 abb.2 abb.2 g1112.b1 abb.3 2002_t . . File_2: 2000_t Ali england 135 abb.1 Zoe british 150 2001_t Ali england 305 g1111.b1 Lucy russia 126

4. Shell Programming and Scripting

Sed or awk : pattern selection based on special characters

Hello All, I am here again scratching my head on pattern selection with special characters. I have a large file having around 200 entries and i have to select a single line based on a pattern. I am able to do that: Code: cat mytest.txt | awk -F: '/myregex/ { print $2}' ...

5. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed...

6. UNIX for Dummies Questions & Answers

using sed delete a line from csv file based on specific data in two separate fields

Hello, :wall: I have a 12 column csv file. I wish to delete the entire line if column 7 = hello and column 12 = goodbye. I have tried everything that I can find in all of my ref books. I know this does not work /^*,*,*,*,*,*,"hello",*,*,*,*,"goodbye"/d Any ideas? Thanks Please...

7. Shell Programming and Scripting

Precede and Append characters using sed/awk based on a pattern

I have an input file which is similar to what I have shown below. Pattern : Data followed by two blank lines followed by data again followed by two blank lines followed by data again etc.. The first three lines after every blank line combination(2 blank lines between data) should be...

8. Shell Programming and Scripting

awk to insert missing string based on pattern in file

Using the file below, which will always have the first indicated by the digit after the - and last id in it, indicated by the digit after the -, I am trying to use awk to print the missing line or lines in file following the pattern of the previous line. For example, in the file below the next...

9. Shell Programming and Scripting

awk to update value based on pattern match in another file

In the awk, thanks you @RavinderSingh13, for the help in below, hopefully it is close as I am trying to update the value in $12 of the tab-delimeted file2 with the matching value in $1 of the space delimeted file1. I have added comments for each line as well. Thank you :). awk awk '$12 ==...

10. Shell Programming and Scripting

Copy data to new file based on input pattern