Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Split file into chunks of low & high byte

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Split file into chunks of low & high byte
# 1  
Old 02-25-2010
Split file into chunks of low & high byte
Hi guys,

i have a question about spliting a binary file into 2 chunks.
First chunk with all high bytes and the second one with all low bytes.

What unix tools can i use? And how can this be performed?
I looked in manpages of split and dd but this does not help.

Thanks
# 2  
Old 02-25-2010
Can you specify what exactly you mean by low and high byte? You want to split alternating bytes in a file into two files? If so, which is the high and which is the low? Is it big endian or little endian. Some sample data and sample desired output is worth 2^10 words Smilie

Alister

---------- Post updated at 01:42 PM ---------- Previous update was at 12:05 PM ----------

In case it's of any use, I went ahead and whipped something up since the problem piqued my interest:

Code:
od -An -vtd1 file | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

Sample run on using a binary file i created which contains 64 bytes whose values increase from 0 (null byte not a problem) to 63 (treating this as bigendian, with the most significant byte first):

Code:
$ od -td1 bin 
0000000     0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
0000020    16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31
0000040    32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47
0000060    48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63
0000100

$ od -An -vtd1 bin | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

$ od -td1 hi
0000000     0   2   4   6   8  10  12  14  16  18  20  22  24  26  28  30
0000020    32  34  36  38  40  42  44  46  48  50  52  54  56  58  60  62
0000040

$ od -td1 lo
0000000     1   3   5   7   9  11  13  15  17  19  21  23  25  27  29  31
0000020    33  35  37  39  41  43  45  47  49  51  53  55  57  59  61  63
0000040

While this may not be an efficient solution (bytes are converted to lines of numeric text which are then converted back to bytes and output one char at a time), it is simple, compact, and should give the desired result. Smilie

Regards,
Alister
Last edited by alister; 02-25-2010 at 03:30 PM..
This User Gave Thanks to alister For This Post:
mzs
# 3  
Old 02-26-2010
Hi alister,

thank you for your reply.

Its just what i needed. I will test it now.

Regards
basta
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies

2. Shell Programming and Scripting

Modification of perl script to split a large file into chunks of 5000 chracters

I have a perl script which splits a large file into chunks.The script is given below use strict; use warnings; open (FH, "<monolingual.txt") or die "Could not open source file. $!"; my $i = 0; while (1) { my $chunk; print "process part $i\n"; open(OUT, ">part$i.log") or die "Could... (4 Replies)
Discussion started by: gimley
4 Replies

3. Shell Programming and Scripting

Combine splitted low & high byte files into one file

Hi all, i have a binary file splitted into 2 chunks, first part with all high bytes and the second part with all low bytes. I need to combine the two chunks into one binary file like (eg. exactly the reverse of the splitting method solved in the thread # 130940) Hi bytes file content:... (7 Replies)
Discussion started by: mzs
7 Replies

4. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

5. Red Hat

High RAM usage, extremely low swapping

Hi team I have three physical servers running on Red Hat Enterprise Linux Server release 6.2 with the following memory conditions: # cat /proc/meminfo | grep -i mem MemTotal: 8062888 kB MemFree: 184540 kB Shmem: 516 kB and the following swap conditions: ... (6 Replies)
Discussion started by: hedkandi
6 Replies

6. AIX

High Runqueue (R) LOW CPU LOW I/O Low Network Low memory usage

Hello All I have a system running AIX 61 shared uncapped partition (with 11 physical processors, 24 Virtual 72GB of Memory) . The output from NMON, vmstat show a high run queue (60+) for continous periods of time intervals, but NO paging, relatively low I/o (6000) , CPU % is 40, Low network.... (9 Replies)
Discussion started by: IL-Malti
9 Replies

7. UNIX for Dummies Questions & Answers

Kernel/ user space and high/ low mem

Need some clarification on this.... 1. how are kernel/ user spaces and high/low memory related? 2. What do they all mean when i have the kernel command line as: "console=ttyS0,115200 root=/dev/sda2 rw mem=exactmap memmap=1M@0 memmap=96M@1M irqpoll" or 2. what do mem and memmap mean in... (3 Replies)
Discussion started by: dragonpoint
3 Replies

8. Shell Programming and Scripting

low & high values

on the file Ftp'd from the mainframe ,do we have any UNIX command to replace mainframe low and values to space or null. i tried using tr and it doesn't work ... Thanks (1 Reply)
Discussion started by: rlmadhav
1 Replies

9. UNIX for Dummies Questions & Answers

malloc returning NULL if freemem high & swapmem low (MPRAS version 3.03 )

Hi All,:) In my application malloc is returning NULL even though there is sufficient amount of free memory available but the swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ? Few details: ... (4 Replies)
Discussion started by: Ritesh Kumar
4 Replies

10. Solaris

malloc returning NULL if freemem high & swapmem low

Hi All, In my application malloc is returning NULL even though there is sufficient amount of free memory is available but swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ?:) Kindly look into... (5 Replies)
Discussion started by: Ritesh Kumar
5 Replies
Login or Register to Ask a Question