Sponsored Content
Top Forums Shell Programming and Scripting how to get data from hex file using SED or AWK based on pattern sign Post 302564018 by alister on Wednesday 12th of October 2011 03:16:37 PM
Old 10-12-2011
Since we don't know exactly what platform you're running on, my proposal endeavours to restrict itself to ubiquitous POSIX functionality. Also, it makes the same assumptions you've made. Specifically:

Quote:
Originally Posted by sameucho
I simplified the task supposing there will not be another occurence of such combination of 'sign' bytes before the desired values which are to be collected.
I did not test the following code, but I did my best to mind the details. If it doesn't work, please post any error messages, how the behavior deviates from what's expected, and which operating system(s) this needs to run on. Also, if the following code is insufficient, it would help to have a sample of the binary data to test against (upload it somewhere and link us). I'm feeling a bit lazy today and I'm not interested in creating my own mock data Smilie (although I suppose I could reverse the hexdump with AWK if I were feeling industrious).


Code:
od -An -td1 binfile | tr -s ' \t' '\n\n' | awk '
    NR==1 && length==0   { getline }
    $0==180              { i=1; getline; getline; getline }
    i==1 && $0==128      { ++i; getline }
    i==2 && $0==1        { ++i; getline }
    i==3 && $0==12       { ++i; getline }
    i==4 && $0==175      { ++i; getline }
    i==5 && $0==131      { ++i; getline; pr_bytes(); getline }
    i==6 && $0==132      { getline; pr_bytes(); printf("%s", s) }
    i!=4 && i!=5         { i=0; s="" }

    function pr_bytes() {
        j=$0
        while (j--) {
                getline
                s=s sprintf("%.2X%s", $0, (j ? OFS : ORS))
        }
    }
'


Since AWK is not required to support hexadecimal constants or numeric strings, od dumps byte values in base 10. tr is used to replace all spaces and tabs with newlines. AWK then reads one line at a time, with each line either containing one byte value in decimal or nothing at all.

The AWK script:
* Discard a leading blank line if present (a by-product of leading whitespace in od output).
* i keeps track of which state is sought.
* pr_bytes() reads the value of the current byte and reads that many subsequent bytes. The bytes are stored in s as a space-delimited string terminated by a newline.
* If at any point a byte value does not match what's expected, the line will fallthrough to the bottom, where i and s are reset.
* The output is two lines of text per record. Line 1 corresponds to what you've referred to as X, line 2 to Y. Each line is a space-delimited sequence of hexadecimal byte values.

Regards,
Alister

Last edited by alister; 10-12-2011 at 04:45 PM.. Reason: Added missing getline and corrected conditional
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Split a file based on pattern in awk, grep, sed or perl

Hi All, Can someone please help me write a script for the following requirement in awk, grep, sed or perl. Buuuu xxx bbb Kmmmm rrr ssss uuuu Kwwww zzzz ccc Roooowwww eeee Bxxxx jjjj dddd Kuuuu eeeee nnnn Rpppp cccc vvvv cccc Rhhhhhhyyyy tttt Lhhhh rrrrrssssss Bffff mmmm iiiii Ktttt... (5 Replies)
Discussion started by: kumarn
5 Replies

2. Shell Programming and Scripting

Truncating FILE data BASED ON A PATTERN

HI I HAVE A PROBLEM,MY SOURCE FILE IS OF PATTERN S1,E-Certified,29,29,2.7,Certified,4,3,2.7,,0,0,0 S2,Certified,4,3,2.7,,0,0,0,,0 S3,E-Certified,29,29,2.7,,0,0,0 S4,,0,0,0,,0,0,0,,0,0,0,,0,0,0 AND THE EXPECTED OUTPUT IS S1,E-Certified,29,29,2.7 S1,Certified,4,3,2.7... (1 Reply)
Discussion started by: pkumar3
1 Replies

3. Shell Programming and Scripting

Merge two file data together based on specific pattern match

My input: File_1: 2000_t g1110.b1 abb.1 2001_t g1111.b1 abb.2 abb.2 g1112.b1 abb.3 2002_t . . File_2: 2000_t Ali england 135 abb.1 Zoe british 150 2001_t Ali england 305 g1111.b1 Lucy russia 126 (6 Replies)
Discussion started by: patrick87
6 Replies

4. Shell Programming and Scripting

Sed or awk : pattern selection based on special characters

Hello All, I am here again scratching my head on pattern selection with special characters. I have a large file having around 200 entries and i have to select a single line based on a pattern. I am able to do that: Code: cat mytest.txt | awk -F: '/myregex/ { print $2}' ... (6 Replies)
Discussion started by: usha rao
6 Replies

5. Shell Programming and Scripting

sed/awk : how to delete lines based on IP pattern ?

Hi, I would like to delete lines in /etc/hosts on few workstations, basically I want to delete all the lines for a list of machines like this : for HOST in $(cat stations.lst |uniq) do # echo -n "$HOST" if ping -c 1 $HOST > /dev/null 2>&1 then HOSTNAME_val=`rsh $HOST "sed... (3 Replies)
Discussion started by: albator1932
3 Replies

6. UNIX for Dummies Questions & Answers

using sed delete a line from csv file based on specific data in two separate fields

Hello, :wall: I have a 12 column csv file. I wish to delete the entire line if column 7 = hello and column 12 = goodbye. I have tried everything that I can find in all of my ref books. I know this does not work /^*,*,*,*,*,*,"hello",*,*,*,*,"goodbye"/d Any ideas? Thanks Please... (2 Replies)
Discussion started by: Chris Eagleson
2 Replies

7. Shell Programming and Scripting

Precede and Append characters using sed/awk based on a pattern

I have an input file which is similar to what I have shown below. Pattern : Data followed by two blank lines followed by data again followed by two blank lines followed by data again etc.. The first three lines after every blank line combination(2 blank lines between data) should be... (2 Replies)
Discussion started by: bikerboy
2 Replies

8. Shell Programming and Scripting

awk to insert missing string based on pattern in file

Using the file below, which will always have the first indicated by the digit after the - and last id in it, indicated by the digit after the -, I am trying to use awk to print the missing line or lines in file following the pattern of the previous line. For example, in the file below the next... (4 Replies)
Discussion started by: cmccabe
4 Replies

9. Shell Programming and Scripting

awk to update value based on pattern match in another file

In the awk, thanks you @RavinderSingh13, for the help in below, hopefully it is close as I am trying to update the value in $12 of the tab-delimeted file2 with the matching value in $1 of the space delimeted file1. I have added comments for each line as well. Thank you :). awk awk '$12 ==... (10 Replies)
Discussion started by: cmccabe
10 Replies

10. Shell Programming and Scripting

Copy data to new file based on input pattern

Hi All, I want to create a new file based on certain conditions and copy only those conditioned data to new file. Input Data is as it looks below. ORDER|Header|Add|32|32|1616 ORDER|Details1......... ORDER|Details2......... ORDER|Details3......... ORDER|Details4............ (10 Replies)
Discussion started by: grvk101
10 Replies
GETLINE(3)						     Linux Programmer's Manual							GETLINE(3)

NAME
getline, getdelim - delimited string input SYNOPSIS
#include <stdio.h> ssize_t getline(char **lineptr, size_t *n, FILE *stream); ssize_t getdelim(char **lineptr, size_t *n, int delim, FILE *stream); Feature Test Macro Requirements for glibc (see feature_test_macros(7)): getline(), getdelim(): Since glibc 2.10: _POSIX_C_SOURCE >= 200809L || _XOPEN_SOURCE >= 700 Before glibc 2.10: _GNU_SOURCE DESCRIPTION
getline() reads an entire line from stream, storing the address of the buffer containing the text into *lineptr. The buffer is null-termi- nated and includes the newline character, if one was found. If *lineptr is NULL, then getline() will allocate a buffer for storing the line, which should be freed by the user program. (In this case, the value in *n is ignored.) Alternatively, before calling getline(), *lineptr can contain a pointer to a malloc(3)-allocated buffer *n bytes in size. If the buffer is not large enough to hold the line, getline() resizes it with realloc(3), updating *lineptr and *n as necessary. In either case, on a successful call, *lineptr and *n will be updated to reflect the buffer address and allocated size respectively. getdelim() works like getline(), except that a line delimiter other than newline can be specified as the delimiter argument. As with get- line(), a delimiter character is not added if one was not present in the input before end of file was reached. RETURN VALUE
On success, getline() and getdelim() return the number of characters read, including the delimiter character, but not including the termi- nating null byte. This value can be used to handle embedded null bytes in the line read. Both functions return -1 on failure to read a line (including end-of-file condition). ERRORS
EINVAL Bad arguments (n or lineptr is NULL, or stream is not valid). VERSIONS
These functions are available since libc 4.6.27. CONFORMING TO
Both getline() and getdelim() were originally GNU extensions. They were standardized in POSIX.1-2008. EXAMPLE
#define _GNU_SOURCE #include <stdio.h> #include <stdlib.h> int main(void) { FILE *fp; char *line = NULL; size_t len = 0; ssize_t read; fp = fopen("/etc/motd", "r"); if (fp == NULL) exit(EXIT_FAILURE); while ((read = getline(&line, &len, fp)) != -1) { printf("Retrieved line of length %zu : ", read); printf("%s", line); } free(line); exit(EXIT_SUCCESS); } SEE ALSO
read(2), fgets(3), fopen(3), fread(3), gets(3), scanf(3) COLOPHON
This page is part of release 3.44 of the Linux man-pages project. A description of the project, and information about reporting bugs, can be found at http://www.kernel.org/doc/man-pages/. GNU
2010-06-12 GETLINE(3)
All times are GMT -4. The time now is 12:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy