Sponsored Content
Top Forums Shell Programming and Scripting Split file into chunks of low & high byte Post 302398736 by alister on Thursday 25th of February 2010 01:42:33 PM
Old 02-25-2010
Can you specify what exactly you mean by low and high byte? You want to split alternating bytes in a file into two files? If so, which is the high and which is the low? Is it big endian or little endian. Some sample data and sample desired output is worth 2^10 words Smilie

Alister

---------- Post updated at 01:42 PM ---------- Previous update was at 12:05 PM ----------

In case it's of any use, I went ahead and whipped something up since the problem piqued my interest:

Code:
od -An -vtd1 file | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

Sample run on using a binary file i created which contains 64 bytes whose values increase from 0 (null byte not a problem) to 63 (treating this as bigendian, with the most significant byte first):

Code:
$ od -td1 bin 
0000000     0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15
0000020    16  17  18  19  20  21  22  23  24  25  26  27  28  29  30  31
0000040    32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47
0000060    48  49  50  51  52  53  54  55  56  57  58  59  60  61  62  63
0000100

$ od -An -vtd1 bin | tr -cs '0-9' '\n' | awk 'NF{printf("%c",$0) > (++i%2?"hi":"lo")}'

$ od -td1 hi
0000000     0   2   4   6   8  10  12  14  16  18  20  22  24  26  28  30
0000020    32  34  36  38  40  42  44  46  48  50  52  54  56  58  60  62
0000040

$ od -td1 lo
0000000     1   3   5   7   9  11  13  15  17  19  21  23  25  27  29  31
0000020    33  35  37  39  41  43  45  47  49  51  53  55  57  59  61  63
0000040

While this may not be an efficient solution (bytes are converted to lines of numeric text which are then converted back to bytes and output one char at a time), it is simple, compact, and should give the desired result. Smilie

Regards,
Alister

Last edited by alister; 02-25-2010 at 03:30 PM..
This User Gave Thanks to alister For This Post:
 

10 More Discussions You Might Find Interesting

1. Solaris

malloc returning NULL if freemem high & swapmem low

Hi All, In my application malloc is returning NULL even though there is sufficient amount of free memory is available but swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ?:) Kindly look into... (5 Replies)
Discussion started by: Ritesh Kumar
5 Replies

2. UNIX for Dummies Questions & Answers

malloc returning NULL if freemem high & swapmem low (MPRAS version 3.03 )

Hi All,:) In my application malloc is returning NULL even though there is sufficient amount of free memory available but the swap memory is low. Is this possible that, if free memory is high & swap memory is low, malloc will not be able to allocate memory & return NULL ? Few details: ... (4 Replies)
Discussion started by: Ritesh Kumar
4 Replies

3. Shell Programming and Scripting

low & high values

on the file Ftp'd from the mainframe ,do we have any UNIX command to replace mainframe low and values to space or null. i tried using tr and it doesn't work ... Thanks (1 Reply)
Discussion started by: rlmadhav
1 Replies

4. UNIX for Dummies Questions & Answers

Kernel/ user space and high/ low mem

Need some clarification on this.... 1. how are kernel/ user spaces and high/low memory related? 2. What do they all mean when i have the kernel command line as: "console=ttyS0,115200 root=/dev/sda2 rw mem=exactmap memmap=1M@0 memmap=96M@1M irqpoll" or 2. what do mem and memmap mean in... (3 Replies)
Discussion started by: dragonpoint
3 Replies

5. AIX

High Runqueue (R) LOW CPU LOW I/O Low Network Low memory usage

Hello All I have a system running AIX 61 shared uncapped partition (with 11 physical processors, 24 Virtual 72GB of Memory) . The output from NMON, vmstat show a high run queue (60+) for continous periods of time intervals, but NO paging, relatively low I/o (6000) , CPU % is 40, Low network.... (9 Replies)
Discussion started by: IL-Malti
9 Replies

6. Red Hat

High RAM usage, extremely low swapping

Hi team I have three physical servers running on Red Hat Enterprise Linux Server release 6.2 with the following memory conditions: # cat /proc/meminfo | grep -i mem MemTotal: 8062888 kB MemFree: 184540 kB Shmem: 516 kB and the following swap conditions: ... (6 Replies)
Discussion started by: hedkandi
6 Replies

7. Shell Programming and Scripting

Split a large array into small chunks

Hi, I need to split a large array "@sharedArray" into 10 small arrays. The arrays should be like @sharedArray1,@sharedArray2,@sharedArray3...so on.. Can anyone help me with the logic to do so :(:confused: (6 Replies)
Discussion started by: rkrish
6 Replies

8. Shell Programming and Scripting

Combine splitted low & high byte files into one file

Hi all, i have a binary file splitted into 2 chunks, first part with all high bytes and the second part with all low bytes. I need to combine the two chunks into one binary file like (eg. exactly the reverse of the splitting method solved in the thread # 130940) Hi bytes file content:... (7 Replies)
Discussion started by: mzs
7 Replies

9. Shell Programming and Scripting

Modification of perl script to split a large file into chunks of 5000 chracters

I have a perl script which splits a large file into chunks.The script is given below use strict; use warnings; open (FH, "<monolingual.txt") or die "Could not open source file. $!"; my $i = 0; while (1) { my $chunk; print "process part $i\n"; open(OUT, ">part$i.log") or die "Could... (4 Replies)
Discussion started by: gimley
4 Replies

10. UNIX for Beginners Questions & Answers

Split large file into smaller files without disturbing the entry chunks

Dears, Need you help with the below file manipulation. I want to split the file into 8 smaller files but without cutting/disturbing the entries (meaning every small file should start with a entry and end with an empty line). It will be helpful if you can provide a one liner command for this... (12 Replies)
Discussion started by: Kamesh G
12 Replies
term(4) 							   File Formats 							   term(4)

NAME
term - format of compiled term file SYNOPSIS
/usr/share/lib/terminfo/?/* DESCRIPTION
The term file is compiled from terminfo(4) source files using tic(1M). Compiled files are organized in a directory hierarchy under the first letter of each terminal name. For example, the vt100 file would have the pathname /usr/lib/terminfo/v/vt100. The default directory is /usr/share/lib/terminfo. Synonyms for the same terminal are implemented by multiple links to the same compiled file. The format has been chosen so that it is the same on all hardware. An 8-bit byte is assumed, but no assumptions about byte ordering or sign extension are made. Thus, these binary terminfo files can be transported to other hardware with 8-bit bytes. Short integers are stored in two 8-bit bytes. The first byte contains the least significant 8 bits of the value, and the second byte con- tains the most significant 8 bits. (Thus, the value represented is 256*second+first.) The value -1 is represented by 0377,0377, and the value -2 is represented by 0376,0377; other negative values are illegal. The -1 generally means that a capability is missing from this ter- minal. The -2 means that the capability has been cancelled in the terminfo source and also is to be considered missing. The compiled file is created from the source file descriptions of the terminals (see the -I option of infocmp) by using the terminfo com- piler, tic, and read by the routine setupterm (see curses(3CURSES)). The file is divided into six parts in the following order: the header, terminal names, boolean flags, numbers, strings, and string table. The header section begins the file six short integers in the format described below. These integers are: 1. the magic number (octal 0432); 2. the size, in bytes, of the names section; 3. the number of bytes in the boolean section 4. the number of short integers in the numbers section; 5. the number of offsets (short integers) in the strings section; 6. the size, in bytes, of the string table. The terminal name section comes next. It contains the first line of the terminfo description, listing the various names for the terminal, separated by the bar ( | ) character (see term(5)). The section is terminated with an ASCII NUL character. The terminal name section is followed by the Boolean section, number section, string section, and string table. The boolean flags section consists of one byte for each flag. This byte is either 0 or 1 as the flag is present or absent. The value of 2 means that the flag has been cancelled. The capabilities are in the same order as the file <term.h>. Between the boolean flags section and the number section, a null byte is inserted, if necessary, to ensure that the number section begins on an even byte offset. All short integers are aligned on a short word boundary. The numbers section is similar to the boolean flags section. Each capability takes up two bytes, and is stored as a short integer. If the value represented is -1 or -2, the capability is taken to be missing. The strings section is also similar. Each capability is stored as a short integer, in the format above. A value of -1 or -2 means the capa- bility is missing. Otherwise, the value is taken as an offset from the beginning of the string table. Special characters in ^X or c nota- tion are stored in their interpreted form, not the printing representation. Padding information ($<nn>) and parameter information (%x) are stored intact in uninterpreted form. The final section is the string table. It contains all the values of string capabilities referenced in the string section. Each string is null terminated. Note that it is possible for setupterm to expect a different set of capabilities than are actually present in the file. Either the database may have been updated since setupterm has been recompiled (resulting in extra unrecognized entries in the file) or the program may have been recompiled more recently than the database was updated (resulting in missing entries). The routine setupterm must be prepared for both possibilities--this is why the numbers and sizes are included. Also, new capabilities must always be added at the end of the lists of boolean, number, and string capabilities. As an example, here is terminal information on the AT&T Model 37 KSR terminal as output by the infocmp -I tty37 command: 37|tty37|AT&T model 37 teletype, hc, os, xon, bel=^G, cr= , cub1=, cud1= , cuu1=E7, hd=E9, hu=E8, ind= , The following is an octal dump of the corresponding term file, produced by the od -c /usr/share/lib/terminfo/t/tty37 command: 0000000 032 001 032 013 021 001 3 3 7 | t 0000020 t y 3 7 | A T & T m o d e l 0000040 3 7 t e l e t y p e 0000060 001 001 0000100 001 377 377 377 377 377 377 377 377 377 377 0000120 377 377 377 377 377 377 377 377 377 377 377 377 377 377 & 0000140 377 377 377 377 377 377 377 377 377 377 377 377 377 377 0000160 377 377 " 377 377 377 377 ( 377 377 377 377 377 377 0000200 377 377 0 377 377 377 377 377 377 377 377 - 377 377 0000220 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 * 0000520 377 377 377 377 377 377 377 377 377 377 377 377 377 377 $ 0000540 377 377 377 377 377 377 377 377 377 377 377 377 377 377 * 0000560 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 377 * 0001160 377 377 377 377 377 377 377 377 377 377 377 377 377 377 3 7 0001200 | t t y 3 7 | A T & T m o d e 0001220 l 3 7 t e l e t y p e 0001240 007  033 8 033 9 033 7 0001260 0001261 Some limitations: total compiled entries cannot exceed 4096 bytes; all entries in the name field cannot exceed 128 bytes. FILES
/usr/share/lib/terminfo/?/* compiled terminal description database /usr/include/term.h terminfo header /usr/xpg4/include/term.h X/Open Curses terminfo header SEE ALSO
infocmp(1M), curses(3CURSES), curses(3XCURSES), terminfo(4), term(5) SunOS 5.10 3 Jul 1996 term(4)
All times are GMT -4. The time now is 07:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy