Sponsored Content
Top Forums Shell Programming and Scripting Extract sequences of bytes from binary for differents blocks Post 302845007 by wisecracker on Tuesday 20th of August 2013 02:36:12 PM
Old 08-20-2013
This is a starter Python 3.3.x code to extract the blocks using a user function...

This should be more than enough to get you going with your binary manipulation.

Once each block is placed into its respective _variable_ it is seriously easy to extract sub-blocks of data from each major block extracted from the binary file...
Code:
# BinarySearch.py
# DEMO to extract binary data...
# Written in such a way as to be easily understood.
# NOTE:- There are more Pythonic ways to do this...

import os
import sys

# The _variable_ used to test with...
block_one=b"?"
block_two=b"?"
subscript_begin=0

# Set to a _temporary_ directory. NOTE:- 4 different platforms shown...
# Change to suit your platform and path
if sys.platform=="linux2": os.chdir("/tmp/")
if sys.platform=="darwin": os.chdir("/Users/barrywalker/")
if sys.platform=="win32": os.chdir("C:\\Windows\\Temp\\")
if sys.platform=="amiga": os.chdir("T:")

# A basic "bin_extract()" function to extract binary data from a binary file...
def bin_extract(subscript_start=0, subscript_end=0, filename="?", char_start=b"\x00\xFF", char_end=b"\xFF\x00"):
	bin_var=b""
	toggle=0
	binfile=open(filename, "rb")
	for position in range(subscript_start, subscript_end, 1):
		binfile.seek(position)
		if binfile.read(2)==char_start:
			bin_var=bin_var+char_start[1:2]
			toggle=1
		if toggle>=1: bin_var=bin_var+binfile.read(1)
		if binfile.read(2)==char_end:
			bin_var=bin_var+char_end[0:1]
			toggle=0
			break
	binfile.close()
	return(bin_var, position)

# Test...
# ******** Do the first iteration and find the first block... *******
block_one, subscript_begin=bin_extract(0, 816, "BinaryFile", b"\xFF\x32", b"\xFF\x34")
print(block_one)
print(subscript_begin)
# When successful..........
# Just extract the required data from the _variable_ "new_binary" using standard simple Python methods
# and when finished with set new values to the function for the next block run using "subscript_begin"
# + or - 2 as the next "subscript_start" in the function, something like below...
block_two, subscript_begin=bin_extract((subscript_begin+2), 816, "BinaryFile", b"\xFF\x32", b"\xFF\x34")
print(block_two)
print(subscript_begin)
# DEMO end...

Hope this helps.
CYA.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies

2. UNIX for Advanced & Expert Users

Deal with binary sequences

Hello, I have come across the necessity for me to deal with binary sequences and I had a few questions. 1- Does any UNIX scripting language provide any tool or command for converting text data to binary sequences? Example of binary sequence: "0x97 0x93 0x85 0x40 0xd5 0xd6 0xd7" 2- If I want... (2 Replies)
Discussion started by: Indalecio
2 Replies

3. Shell Programming and Scripting

Extract sequence blocks

Hi, I have an one-line file consisting of a sequence of 660 letters. I would like to extract 9-letter blocks iteratively: ASDFGHJKLQWERTYUIOPZXCVBNM first block: ASDFGHJKL 1nd block: SDFGHJKLQ What I have so far only gives me the first block, can anyone please explain why? cat... (7 Replies)
Discussion started by: solli
7 Replies

4. Shell Programming and Scripting

extract blocks of text from a file

Hi, This is part of a large text file I need to separate out. I'd like some help to build a shell script that will extract the text between sets of dashed lines, write that to a new file using the whole or part of the first text string as the new file name, then move on to the next one and... (7 Replies)
Discussion started by: cajunfries
7 Replies

5. Linux

Why does ext3 allocate 8 blocks for files that are few bytes long

The title is clear: why does ext3 allocate 8 blocks for files that are few bytes long? If I create a file named "test", put a few chars in it, and then I run: stat test I get that "Blocks: 8" I searched in the web and found that ext does that, it allocates 8 blocks even if It doesn't need... (4 Replies)
Discussion started by: Tavo
4 Replies

6. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

7. Shell Programming and Scripting

Extract sequences based on the list

Hi, I have a file with more than 28000 records and it looks like below.. >mm10_refflat_ABCD range=chr1:1234567-2345678 tgtgcacactacacatgactagtacatgactagac....so on >mm10_refflat_BCD range=chr1:3234567-4545678... tgtgcacactacacatgactagtatgtgcacactacacatgactagta . . . . . so on ... (2 Replies)
Discussion started by: Diya123
2 Replies

8. Shell Programming and Scripting

Extract length wise sequences from fastq file

I have a fastq file from small RNA sequencing with sequence lengths between 15 - 30. I wanted to filter sequence lengths between 21-25 and write to another fastq file. how can i do that? (4 Replies)
Discussion started by: empyrean
4 Replies

9. Shell Programming and Scripting

Extract the part of sequences from a file

I have a text file, input.fasta contains some protein sequences. input.fasta is shown below. >P02649 MKVLWAALLVTFLAGCQAKVEQAVETEPEPELRQQTEWQSGQRWELALGRFWDYLRWVQT LSEQVQEELLSSQVTQELRALMDETMKELKAYKSELEEQLTPVAEETRARLSKELQAAQA RLGADMEDVCGRLVQYRGEVQAMLGQSTEELRVRLASHLRKLRKRLLRDADDLQKRLAVY... (8 Replies)
Discussion started by: rahim42
8 Replies

10. Shell Programming and Scripting

Blocks of text in a file - extract when matches...

I sat down yesterday to write this script and have just realised that my methodology is broken........ In essense I have..... ----------------------------------------------------------------- (This line really is in the file) Service ID: 12345 ... (7 Replies)
Discussion started by: Bashingaway
7 Replies
io_tryread(3)						     Library Functions Manual						     io_tryread(3)

NAME
io_tryread - read from a descriptor without blocking SYNTAX
#include <io.h> int io_tryread(int64 fd,char* buf,int64 len); DESCRIPTION
io_tryread tries to read len bytes of data from descriptor fd into buf[0], buf[1], ..., buf[len-1]. (The effects are undefined if len is 0 or smaller.) There are several possible results: o o_tryread returns an integer between 1 and len: This number of bytes was available for immediate reading; the bytes were read into the beginning of buf. Note that this number can be, and often is, smaller than len; you must not assume that io_tryread always succeeds in reading exactly len bytes. o io_tryread returns 0: No bytes were read, because the descriptor is at end of file. For example, this descriptor has reached the end of a disk file, or is reading an empty pipe that has been closed by all writers. o io_tryread returns -1, setting errno to EAGAIN: No bytes were read, because the descriptor is not ready. For example, the descriptor is reading an empty pipe that could still be written to. o io_tryread returns -3, setting errno to something other than EAGAIN: No bytes were read, because the read attempt encountered a persis- tent error, such as a serious disk failure (EIO), an unreachable network (ENETUNREACH), or an invalid descriptor number (EBADF). io_tryread does not pause waiting for a descriptor that is not ready. If you want to pause, use io_waitread or io_wait. You can make io_tryread faster and more efficient by making the socket non-blocking with io_nonblock(). SEE ALSO
io_nonblock(3), io_waitread(3), io_tryreadtimeout(3) io_tryread(3)
All times are GMT -4. The time now is 10:21 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy