Sponsored Content
Top Forums Shell Programming and Scripting Split file into chunks of low & high byte Post 302398736 by alister on Thursday 25th of February 2010 01:42:33 PM
Old 02-25-2010
Can you specify what exactly you mean by low and high byte? You want to split alternating bytes in a file into two files? If so, which is the high and which is the low? Is it big endian or little endian. Some sample data and sample desired output is worth 2^10 words Smilie

Alister

---------- Post updated at 01:42 PM ---------- Previous update was at 12:05 PM ----------

In case it's of any use, I went ahead and whipped something up since the problem piqued my interest:

Code:
od -An -vtd1 file | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

Sample run on using a binary file i created which contains 64 bytes whose values increase from 0 (null byte not a problem) to 63 (treating this as bigendian, with the most significant byte first):

Code:
$ od -td1 bin 
0000000     0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
0000020    16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31
0000040    32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47
0000060    48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63
0000100

$ od -An -vtd1 bin | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

$ od -td1 hi
0000000     0   2   4   6   8  10  12  14  16  18  20  22  24  26  28  30
0000020    32  34  36  38  40  42  44  46  48  50  52  54  56  58  60  62
0000040

$ od -td1 lo
0000000     1   3   5   7   9  11  13  15  17  19  21  23  25  27  29  31
0000020    33  35  37  39  41  43  45  47  49  51  53  55  57  59  61  63
0000040

While this may not be an efficient solution (bytes are converted to lines of numeric text which are then converted back to bytes and output one char at a time), it is simple, compact, and should give the desired result. Smilie

Regards,
Alister

Last edited by alister; 02-25-2010 at 03:30 PM..
This User Gave Thanks to alister For This Post:
 

10 More Discussions You Might Find Interesting

1. Solaris

malloc returning NULL if freemem high & swapmem low

Hi All, In my application malloc is returning NULL even though there is sufficient amount of free memory is available but swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ?:) Kindly look into... (5 Replies)
Discussion started by: Ritesh Kumar
5 Replies

2. UNIX for Dummies Questions & Answers

malloc returning NULL if freemem high & swapmem low (MPRAS version 3.03 )

Hi All,:) In my application malloc is returning NULL even though there is sufficient amount of free memory available but the swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ? Few details: ... (4 Replies)
Discussion started by: Ritesh Kumar
4 Replies

3. Shell Programming and Scripting

low & high values

on the file Ftp'd from the mainframe ,do we have any UNIX command to replace mainframe low and values to space or null. i tried using tr and it doesn't work ... Thanks (1 Reply)
Discussion started by: rlmadhav
1 Replies

4. UNIX for Dummies Questions & Answers

Kernel/ user space and high/ low mem

Need some clarification on this.... 1. how are kernel/ user spaces and high/low memory related? 2. What do they all mean when i have the kernel command line as: "console=ttyS0,115200 root=/dev/sda2 rw mem=exactmap memmap=1M@0 memmap=96M@1M irqpoll" or 2. what do mem and memmap mean in... (3 Replies)
Discussion started by: dragonpoint
3 Replies

5. AIX

High Runqueue (R) LOW CPU LOW I/O Low Network Low memory usage

Hello All I have a system running AIX 61 shared uncapped partition (with 11 physical processors, 24 Virtual 72GB of Memory) . The output from NMON, vmstat show a high run queue (60+) for continous periods of time intervals, but NO paging, relatively low I/o (6000) , CPU % is 40, Low network.... (9 Replies)
Discussion started by: IL-Malti
9 Replies

6. Red Hat

High RAM usage, extremely low swapping

Hi team I have three physical servers running on Red Hat Enterprise Linux Server release 6.2 with the following memory conditions: # cat /proc/meminfo | grep -i mem MemTotal: 8062888 kB MemFree: 184540 kB Shmem: 516 kB and the following swap conditions: ... (6 Replies)
Discussion started by: hedkandi
6 Replies

7. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

8. Shell Programming and Scripting

Combine splitted low & high byte files into one file

Hi all, i have a binary file splitted into 2 chunks, first part with all high bytes and the second part with all low bytes. I need to combine the two chunks into one binary file like (eg. exactly the reverse of the splitting method solved in the thread # 130940) Hi bytes file content:... (7 Replies)
Discussion started by: mzs
7 Replies

9. Shell Programming and Scripting

Modification of perl script to split a large file into chunks of 5000 chracters

I have a perl script which splits a large file into chunks.The script is given below use strict; use warnings; open (FH, "<monolingual.txt") or die "Could not open source file. $!"; my $i = 0; while (1) { my $chunk; print "process part $i\n"; open(OUT, ">part$i.log") or die "Could... (4 Replies)
Discussion started by: gimley
4 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
OD(1)								   User Commands							     OD(1)

NAME
od - dump files in octal and other formats SYNOPSIS
od [OPTION]... [FILE]... od [-abcdfilosx]... [FILE] [[+]OFFSET[.][b]] od --traditional [OPTION]... [FILE] [[+]OFFSET[.][b] [+][LABEL][.][b]] DESCRIPTION
Write an unambiguous representation, octal bytes by default, of FILE to standard output. With more than one FILE argument, concatenate them in the listed order to form the input. With no FILE, or when FILE is -, read standard input. If first and second call formats both apply, the second format is assumed if the last operand begins with + or (if there are 2 operands) a digit. An OFFSET operand means -j OFFSET. LABEL is the pseudo-address at first byte printed, incremented when dump is progressing. For OFFSET and LABEL, a 0x or 0X prefix indicates hexadecimal; suffixes may be . for octal and b for multiply by 512. Mandatory arguments to long options are mandatory for short options too. -A, --address-radix=RADIX output format for file offsets; RADIX is one of [doxn], for Decimal, Octal, Hex or None --endian={big|little} swap input bytes according the specified order -j, --skip-bytes=BYTES skip BYTES input bytes first -N, --read-bytes=BYTES limit dump to BYTES input bytes -S BYTES, --strings[=BYTES] output strings of at least BYTES graphic chars; 3 is implied when BYTES is not specified -t, --format=TYPE select output format or formats -v, --output-duplicates do not use * to mark line suppression -w[BYTES], --width[=BYTES] output BYTES bytes per output line; 32 is implied when BYTES is not specified --traditional accept arguments in third form above --help display this help and exit --version output version information and exit Traditional format specifications may be intermixed; they accumulate: -a same as -t a, select named characters, ignoring high-order bit -b same as -t o1, select octal bytes -c same as -t c, select printable characters or backslash escapes -d same as -t u2, select unsigned decimal 2-byte units -f same as -t fF, select floats -i same as -t dI, select decimal ints -l same as -t dL, select decimal longs -o same as -t o2, select octal 2-byte units -s same as -t d2, select decimal 2-byte units -x same as -t x2, select hexadecimal 2-byte units TYPE is made up of one or more of these specifications: a named character, ignoring high-order bit c printable character or backslash escape d[SIZE] signed decimal, SIZE bytes per integer f[SIZE] floating point, SIZE bytes per float o[SIZE] octal, SIZE bytes per integer u[SIZE] unsigned decimal, SIZE bytes per integer x[SIZE] hexadecimal, SIZE bytes per integer SIZE is a number. For TYPE in [doux], SIZE may also be C for sizeof(char), S for sizeof(short), I for sizeof(int) or L for sizeof(long). If TYPE is f, SIZE may also be F for sizeof(float), D for sizeof(double) or L for sizeof(long double). Adding a z suffix to any type displays printable characters at the end of each output line. BYTES is hex with 0x or 0X prefix, and may have a multiplier suffix: b 512 KB 1000 K 1024 MB 1000*1000 M 1024*1024 and so on for G, T, P, E, Z, Y. EXAMPLES
od -A x -t x1z -v Display hexdump format output od -A o -t oS -w16 The default output format used by od AUTHOR
Written by Jim Meyering. REPORTING BUGS
GNU coreutils online help: <http://www.gnu.org/software/coreutils/> Report od translation bugs to <http://translationproject.org/team/> COPYRIGHT
Copyright (C) 2017 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. SEE ALSO
Full documentation at: <http://www.gnu.org/software/coreutils/od> or available locally via: info '(coreutils) od invocation' GNU coreutils 8.28 January 2018 OD(1)
All times are GMT -4. The time now is 11:18 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy