Login or Register to Ask a Question and Join Our Community


UNIX for Advanced & Expert Users

How to parse a binary file?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users How to parse a binary file?
# 1  
Old 05-16-2013
How to parse a binary file?
Hello to all in forum,

Please your help with this.

I have a binary file that is represented in Binary or Binary Coded Decimal (BCD).

Do you know a parser for this kind of binary files?

I have the structure format, but I don't know how and where to begin.

You answer would be very appreciated.
# 2  
Old 05-17-2013
Take a look at this info:text processing - How to use bash script to read binary file content? - Unix & Linux Stack Exchange

Also post example input and define exactly what you want to do.
# 3  
Old 05-17-2013
Post the definition of the structure format...
# 4  
Old 05-17-2013
Hello spacebar and shamrock,

I want to convert to readable text the binary file to be able to parse it based on the structure defined.

Below a sample input after applying hexdump -C.
Code:
00000000  31 58 58 58 5f 58 58 58  5f 4c 58 58 36 30 39 30  |1XXX_XXX_XX06090|
00000010  37 32 36 30 30 33 31 07  ff ff ff ff ff ff ff ff  |7260031.........|
00000020  32 00 00 01 24 32 58 01  30 08 29 0f 47 42 05 00  |2...p...0.).PB..|
00000030  01 ff ff ff 00 15 00 0a  48 00 01 33 00 01 36 00  |........H..3..6.|
00000040  01 37 00 01 66 00 01 65  00 01 77 00 01 78 00 01  |.7..f..e..w..x..|
00000047  69 00 01 79 00 04 93 00  01 22 00 00 21 00 02 09  |i..y....."..!...|
00000060  00 01 26 00 01 0f 00 01  11 00 01 13 00 01 08 00  |..&.............|
00000070  01 2b 00 00 2c 00 00 2d  00 00 2e 00 00 55 00 01  |.+..,..-.....U..|
00000080  56 00 07 2a 00 00 2f 00  00 30 00 00 31 00 00 ff  |V..*../..0..1...|
00000090  34 00 80 19 32 c9 06 00  00 08 88 a0 e0 ca 0e 54  |4...2..........T|
000000a0  00 91 0b 47 43 39 90 10  8f ff ff 00 14 80 09 35  |...PC9.........5|
000000b0  c9 06 00 00 08 88 00 00  03 81 0f 01 02 00 00 00  |................|
000000c0  01 47 43 99 01 05 ff ff  ff 00 83 10 01 0c 00 00  |.PC.............|
000000d0  00 01 47 43 99 01 05 ff  ff ff 00 01 86 1d 02 0c  |..PC............|
000000e0  00 00 00 04 47 43 99 01  09 ff ff ff 00 0e 00 00  |....PC..........|
000000f0  00 04 47 43 99 01 09 ff  ff ff 00 87 0f 01 01 00  |..PC............|
00000100  00 00 03 47 43 99 01 08  ff ff ff 00 84 0e 00 00  |...PC...........|
00000110  00 00 00 00 01 01 ff ff  00 00 00 00 85 06 00 00  |................|
00000120  00 00 00 00 ff 32 00 00  02 24 32 58 01 30 08 29  |.....2...p...0.)|
00000130  2f 47 42 05 00 03 ff ff  ff 00 15 00 0a 48 00 01  |/PB..........H..|
00000140  33 00 01 36 00 01 37 00  01 66 00 01 65 00 01 77  |3..6..7..f..e..w|
00000147  00 01 78 00 01 69 00 01  79 00 04 93 00 01 22 00  |..x..i..y.....".|
00000160  00 21 00 02 09 00 01 26  00 01 0f 00 01 11 00 01  |.!.....&........|
00000170  13 00 01 08 00 01 2b 00  00 2c 00 00 2d 00 00 2e  |......+..,..-...|
00000180  00 00 55 00 01 56 00 07  2a 00 00 2f 00 00 30 00  |..U..V..*../..0.|
00000190  00 31 00 00 ff 34 00 80  19 32 c9 06 00 00 08 88  |.1...4...2......|
000001a0  a0 e0 ca 0e 54 00 91 0b  47 43 39 90 10 8f ff ff  |....T...PC9.....|
000001b0  00 14 80 09 35 c9 06 00  00 08 88 00 00 03 81 0f  |....5...........|
000001c0  01 02 00 00 00 01 47 43  99 01 05 ff ff ff 00 83  |......PC........|
000001d0  10 01 0c 00 00 00 01 47  43 99 01 05 ff ff ff 00  |.......PC.......|

Part of the structure definition is below.

Code:
Data representation:    Binary, Binary Coded Decimal (BCD), ISO
Numeral presentation:    Left-adjusted with trailing #F (where required) 
Text presentation:    Left-adjusted with trailing spaces (where required)
Filler:    Spaces for ISO characters and #F for BCD numbers and TBCD string

0    1    Numeral 1    Record type, administrative data 
ISO coded.
1    12    Identifier 1 - 12 characters    Exchange Identity 
12 most significant characters in the exchange identity.
ISO coded.
13    2    Digit string 00 - 99    Starting year for recording 
ISO coded.
15    2    Digit string 01 - 12    Starting month for recording 
ISO coded.
17    2    Digit string 01 - 31    Starting day for recording 
ISO coded.
19    2    Digit string 00 - 23    Starting hour for recording 
ISO coded.
21    2    Digit string 00 - 59    Starting minute for recording 
ISO coded.

Thanks for any advice.

---------- Post updated at 10:50 PM ---------- Previous update was at 12:29 AM ----------

Maybe somebody can advise.

I only want to know if someone has used some binary parser that let me put in a template for each parameter within the file the number of bytes, type of encoding of each byte etc... I mean, put de description of each byte of groups of bytes in a template to be able to parse it.

Thanks for any help.
Last edited by Scott; 05-21-2013 at 04:32 AM.. Reason: More code tags
# 5  
Old 05-18-2013
Do you have the 'struct' or template?
Here is an example in perl:Perl pack and unpack : joakimbech.com
# 6  
Old 05-18-2013
Hello spacebar,

Thanks for your help and for share that link, I'll check it!

The structure description is very long. But all is almost the same as the follwing description of the firsts bytes.


Code:
Position........Bytes........Value.....................................Field Description[/COLOR]
0...................1...............Numeral 1..............................Record type, administrative data ISO coded.
1...................12..............Identifier 1 - 12 characters.....Site Identity (12 most significant characters in the site identity, ISO coded.)
13..................2...............Digit string 00 - 99.................Starting year for recording ISO coded.
15..................2...............Digit string 01 - 12.................Starting month for recording ISO coded.
17..................2...............Digit string 01 - 31.................Starting day for recording ISO coded.
19..................2...............Digit string 00 - 23.................Starting hour for recording ISO coded.
21..................2...............Digit string 00 - 59.................Starting minute for recording ISO coded.

Last edited by Scott; 05-21-2013 at 04:33 AM.. Reason: Code tags not font and colour, please!
# 7  
Old 05-18-2013
Hi Ophicus...

Code:
00000000  31 58 58 58 5f 58 58 58  5f 4c 58 58 36 30 39 30  |1XXX_XXX_XX06090|

This does NOT make sense...

Have you manually altered ths?

Bytes 0 to 8 are correct...
Byte 9 is 0x4C and should read "L"...
Byte 10 is correct...
Byte 11 should read "X"...

However try this to extract a single binary byte from anywhere in the file then you can work on it...

Code:
https://www.unix.com/shell-programming-scripting/212715-another-building-block-binary-file-manipulation.html
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Convert binary file to csv and then back to the binary format

Hello *nix specialists, Im working for a non profit organisation in Germany to transport DSL over WLAN to people in areas without no DSL. We are using Linksys WRT 54 router with DD-WRT firmware There are at the moment over 180 router running but we have to change some settings next time. So my... (7 Replies)
Discussion started by: digidax
7 Replies

2. Shell Programming and Scripting

Python Binary File Read and Parse

Hi to everyone :), i have a challenge right now in python that for now needs a bit of help in one part of the c0de. The task is create a new file with the name of the file defined by the ASCII content between the 3 byte and the 16 byte that is parsed from the binary file, the file is over 20 Mb i... (0 Replies)
Discussion started by: drd0spt
0 Replies

3. UNIX for Dummies Questions & Answers

[AIX] Binary file warning for text file.

Hello guys, We had to move from a DC to another, and we are now facing an "issue" with some text files. Looks like that some of our log files are set as binary: file TuxConnectorURA.20121012 TuxConnectorURA.20121012: data or International Language text less TuxConnectorURA.20121012... (2 Replies)
Discussion started by: EnioMarques
2 Replies

4. Shell Programming and Scripting

Output redirection of c binary file to a file in shell script is failing

I am struck up with a problem and that is with output redirection. I used all the ways for the redirection of the output of c binary to a file, still it is failing. Here are the different ways which I have used: ./a.out | tee -a /root/tmp.txt 2>&1 ./a.out | tee -a /root/tmp.txt 1>&1 ./a.out |... (2 Replies)
Discussion started by: Maya29988
2 Replies

5. UNIX for Advanced & Expert Users

How to copy a binary file while the file is being written to by another process

Hello, Can I copy a binary file while the file is being written to by another process? Another process (program) “P1” creates and opens (for writing) binary file “ABC” on local disk. Process P1 continuously write into ABC file every couple of seconds, adding 512-byte blocks of data. ABC file... (1 Reply)
Discussion started by: mbuki
1 Replies

6. UNIX for Dummies Questions & Answers

Pipe binary file matches grep results to file

I am using grep to match a pattern, but the output is strange. $ grep -r -o "pattern" * Gives me: Binary file foo1 matches Binary file foo2 matches Binary file foo3 matches To find the lines before/after, I then have to use the following on each file: $ strings foo1 | grep -A1 -B1... (0 Replies)
Discussion started by: chipperuga
0 Replies

7. Shell Programming and Scripting

To log binary file output to a txt file

Hi, I wrote a small script whose function is to execute the postemsg provided if the threshold breaches. I want to log this postemsg messages to a log file. But I am not able to do. Can someone throw some light on how to log the output of this. I am pasting a snippet of that code. ... (2 Replies)
Discussion started by: dbashyam
2 Replies

8. UNIX for Dummies Questions & Answers

Binary file

How can we see the contents of a binary file? (1 Reply)
Discussion started by: siba.s.nayak
1 Replies

9. Shell Programming and Scripting

dd - binary file

I have a task that says: make a file (called binaryfile ) that contains 4 bytes of NULL data 6 bytes of random data 8 bytes of 1 10 bytes of 5 and 12 bytes of 9. For the first 2, I can used : dd if=/dev/null of=binaryfile bs=8 count =4 and dd if=/dev/urandom of=binaryfile bs=8... (3 Replies)
Discussion started by: spiriad
3 Replies

10. Solaris

compiled binary file gives "cannot execute binary file"

Hi, I have two Solaris machines. 1. SunOS X 5.8 Generic_108528-29 sun4u sparc SUNW,Sun-Blade-1500 2. SunOS Y 5.8 Generic_108528-13 sun4u sparc SUNW,Ultra-60 I am trying to buiild a project on both these machines. The Binary output file compiled on machine 2 runs on both the machines. Where... (0 Replies)
Discussion started by: scgupta
0 Replies
Login or Register to Ask a Question