Read Write byte range/chunk of data from specific location in file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Read Write byte range/chunk of data from specific location in file
# 1  
Old 05-22-2009
Read Write byte range/chunk of data from specific location in file

I am new to Unix so will really appreciate if someone can guide me on this.
What I want to do is:

Step1: Read binary file - pick first 2 bytes, convert from hex to decimal. Read the next 3 bytes as well.

2 bytes will specify the number of bytes 'n' that I want to read and write (starting from location of 6th byte) only if the 3 bytes meet a specific pattern

Step2: If the 3 bytes meet condition/pattern then I want to read next 'n' bytes starting from location of 6th byte. Write these 'n' bytes to output file.

This step should move seek pointer to 5+n byte location
Step3: Read the next 2 bytes - if these match a prespecified pattern ( alternative is to ignore/skip these 2 bytes)



Step 3: read next 2 byte and 3 bytes again. Repeat step1 and step 2. If condition is met then append bytes to output file from step2.

The binary file is:

0000000: 00570101080100ac8b905751485801bb .W........WQHX..
0000010: 8d5c0afdf44304b80000011480a5041d .\...C..........
0000020: 0000092cdd1ddd000f08000006013372 ...,..........3r
0000030: 6240808edea900a000000e7bbead0180 b@.........{....
0000040: 1f720301801f7100f7018029e2017001 .r....q....)..p.
0000050: 801f720100faae009a0101060100ac8b ..r.............
0000060: 90576284b502f8053c0afdf443027800 .Wb.....<...C.x.
0000070: 0001148105050d02100d02422e41010f ...........B.A..
0000080: 08000006013375837080b2ec1d000000 .....3u.p.......
0000090: 0000030d01802e9100016e01801ff200 ..........n.....
00000a0: 003f018022e20001f201802cf1000018 .?.."......,....
00000b0: 01802e1300007e01802f910001350180 ......~../...5..
00000c0: 2cf300002401801f720000c0018022e1 ,...$...r.....".
00000d0: 0001ec01801ff1000039018022e30001 .........9.."...
00000e0: f801802d6300001201802d61000006fa ...-c.....-a....
00000f0: ae01560101020100ac8b905781321202 ..V........W.2..
0000100: f8053d0afdf443027800000114810505 ..=...C.x.......
0000110: 0d02100d02422e41010f080000060133 .....B.A.......3
0000120: 75837080b2ec1d00000006030300ff05 u.p.............
0000130: 000000060000013d0000010e0000013d .......=.......=
0000140: 0a0000000000000301802e9300010180 ................
0000150: 2e930025000001017a01802e93010180 ...%....z.......
0000160: 2e93017a00007000322c6c000000017a ...z..p.2,l....z
0000170: 01802e930201802e93017a1d00670032 ..........z..g.2
0000180: 2c6c01802e92017415006800322c6c06 ,l.....t..h.2,l.
0000190: 06000000000000198f1a0000012d3400 .............-4.
00001a0: 0000004e000000006800000002820000 ...N....h.......
00001b0: 000a9c00000026b600000000d0000000 ......&.........
00001c0: 00ea00000000ff000000000000000000 ................
00001d0: 00ea231a0000000034000000004e0000 ..#.....4....N..
00001e0: 0000680000000082000000009c000000 ..h.............
00001f0: 00b60000000dd00000012dea00000000 ..........-.....
0000200: ff00000000000d000700070201000000 ................
0000210: 0000087db00000014304000000000004 ...}....C.......
0000220: a65b0000001c0201000000000000d79e .[..............
0000230: 00000120040000000000001285000000 ... ............
0000240: 1a00010501faae00500101090100ac8b ........P.......
0000250: 9057950ff902f8053e0afdf443027800 .W......>...C.x.
0000260: 0001148105050d02100e78022e41240f ..........x..A$.
0000270: 0800000228217304704c85681201802f ....(!s.pL.h.../
0000280: 3301802f3300000000000000b5000000 3../3...........
0000290: 0e000000b5faae00a10101060100ac8b ................
00002a0: 905795164502f8053f0afdf443027800 .W..E...?...C.x.
00002b0: 0001148105050d02100e78022e41240f

So I want to read 0057 convert to decimal. This gives me number of bytes 'n' to read.
Then I read next 3 bytes- 010108. If this is equal to say 010102 then I read the next 'n' bytes where 'n' is from hex to decimal conversion of 0057. Write these 'n' bytes to output file.

This will take (seek) to position in file where next 2 bytes are 'faae'. I will ignore these two bytes.

Read the next 2 (after faae) and subsequent 3 bytes again. Test the condition (equal to 010102) and if yes append 'm' bytes to file generated in above step

Repeat this till the end of file.

In short 'faae' indicats end of one chunk and I need to output chunk to file only if it meets condition The size of chunk is variable specified by 2 bytes after faae.

If someone can help me I will really appreciate.

Thanks

Last edited by Kbenipel; 05-22-2009 at 04:29 PM..
# 2  
Old 05-22-2009
No homework.

Forum Rules
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to read all data after a specific string from a text file ?

Hi, I have a file(input.txt) and trying to format as output.txt. See the attached file format. Note: This is a windows file (DOS format) and the commands are also going to execute on windows. Basically I am trying to capture all the data in between Local Group Memberships and Global Group... (10 Replies)
Discussion started by: Monoj2014
10 Replies

2. Shell Programming and Scripting

Write out specific data from log to a new file

I got a huge log in zipped files, i need to write out lines by specific data and if the line with the same contains XML message with the same sessionID will be written to the file to. The log structure: 2013-08-16 16:31:06,810 ( 122: rogate) INFO - UId:10453, GId:5422: new... (16 Replies)
Discussion started by: batka
16 Replies

3. Shell Programming and Scripting

Read user input, Encrypt the data and write to file

Hi, can some one help me how to encrypt and decrypt a file. AIM: reade user input, encrypt it and save it to file. while decryption read the encrypted file decrypt it and save the output in some variable. Example: consider we have Credentials.txt file with content username: password... (5 Replies)
Discussion started by: saichand1985
5 Replies

4. Shell Programming and Scripting

how read specific line in a file and write it in a new text file?

I have list of files in a directory 'dir'. Each file is of type HTML. I need to read each file and get the string which starts with 'http' and write them in a new text file. How can i do this shell scripting? file1.html <head> <url>http://www.google.com</url> </head> file2.html <head>... (6 Replies)
Discussion started by: vel4ever
6 Replies

5. Shell Programming and Scripting

read data from file from a specific duration

Hi, i have log file which keeps on updating almost ever minute but i have certain string in log which will only be displayed between specific time. so i need to search for that string and copy the details starting from that string and stop at certain string below that. example: log file... (2 Replies)
Discussion started by: gpk_newbie
2 Replies

6. Shell Programming and Scripting

Search for a specific data in a file based on a date range

Hi, Currently I am working on a script to automate the process of converting the log file from binary into text format. To achieve this, partly I am depending on my application’s utility for this conversion and the rest I am relying on shell commands to search for directory, locate the file and... (5 Replies)
Discussion started by: svajhala
5 Replies

7. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

8. Shell Programming and Scripting

Write shelll script to read file location

hi all i have a problem how to read file location..I read file as FILE=/home/tmp/new.file.but t is not useful for me.But i want my script read file location where the file is and copy in directory at boot time. Every time of booting files are copied in respective folder.please help !!!!:) (2 Replies)
Discussion started by: shubhig15
2 Replies

9. Shell Programming and Scripting

Read the apecific data from one file and write into another file

Hi, I would like to read the specific data from file and write the data in the new file. My data input is something like this.. <EXROP:R=TJ0311T; ROUTE DATA R ROUTE PARAMETERS TJ0311T DETY=UPDR TTRANS=1 FNC=3 MA=628160955000 R=TJ0311D ... (3 Replies)
Discussion started by: bha148
3 Replies

10. Programming

Read/Write a fairly large amount of data to a file as fast as possible

Hi, I'm trying to figure out the best solution to the following problem, and I'm not yet that much experienced like you. :-) Basically I have to read a fairly large file, composed of "messages" , in order to display all of them through an user interface (made with QT). The messages that... (3 Replies)
Discussion started by: emitrax
3 Replies
Login or Register to Ask a Question