Sponsored Content
Top Forums UNIX for Dummies Questions & Answers ConCATenating binaries but excluding last bytes from each file Post 302877652 by grolido on Monday 2nd of December 2013 06:49:02 PM
Old 12-02-2013
ConCATenating binaries but excluding last bytes from each file

Hi there, shameful Linux Newbie here Smilie

I was wondering if you could help with my problem...
I have plenty of files I'd like to concatenate. I know how to basically use cat command but that won't be enough from what I need : excluding the last xx bytes from files before assembling since there's some redundancy


the interesting part is the xx bytes to remove is determined by the filename themselves, ie


file 1 is named something like 0-54548
file 2 is named something like 54475-648459
file 3 is named 648345-1269494
etc


so for file 1 I would have to drop bytes between 54475 till the end before joining file 2. etc
it would be easier if it was always the exact same bytes value to remove , alas there are some variations, so it must be calculated from filenames.


I feel like it's possible to script something with not much lines within a loop, but my unix knowledge is way too rudimentary for now ^^


I also read about dd command which could probably be of some help to generate the files without the unwanted part, now I'm a bit clueless about extracting character chains from filenames, turning into numeric values, doing the maths and use this dd or whatever else command so I can assemble the resulting files...
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Take a file from the system and put on tape and reset the file to 0 bytes

:mad: I did this the other day but one of my support personnel removed my history so i could call it back up to remeber the exact command since i am air-headed at times. I am trying to take a 30 MEG file off the system and drop it to tape then i want to make the file go back to being 0 bytes so... (1 Reply)
Discussion started by: JackieRyan26
1 Replies

2. Shell Programming and Scripting

Remove first N bytes and last N bytes from a binary file on AIX.

Hi all, Does anybody know or guide me on how to remove the first N bytes and the last N bytes from a binary file? Is there any AWK or SED or any command that I can use to achieve this? Your help is greatly appreciated!! Best Regards, Naveen. (1 Reply)
Discussion started by: naveendronavall
1 Replies

3. Shell Programming and Scripting

Concatenating the two lines in a file

hi My requirement is i have a file with some records like this file name ::xyz a=1 b=100,200 ,300,400 ,500,600 c=700,800 d=900 i want to change my file a=1 b=100,200,300,400 c=700,800 d=900 if record starts with " , " that line should fallows the previous line.please give... (6 Replies)
Discussion started by: srivsn
6 Replies

4. UNIX for Dummies Questions & Answers

Bytes of character in file

Hi, How do I check for the total bytes of character used by a file? Can I used a od command to check? Thanks. (1 Reply)
Discussion started by: user50210
1 Replies

5. Programming

how to inspect the bytes in a file?

What is the easiest way to inspect the bytes stored in a file? Ideally, If my file was 10 bytes each of which had only the high bit set, I'd be able to browse for it and get output like this: 01 - 10000000 02 - 10000000 03 - 10000000 04 - 10000000 05 - 10000000 06 - 10000000 07 -... (7 Replies)
Discussion started by: sneakyimp
7 Replies

6. Shell Programming and Scripting

Error PHP Fatal error: Allowed memory size of 67108864 bytes exhausted(tried to allocate 401 bytes)

While running script I am getting an error like Few lines in data are not being processed. After googling it I came to know that adding such line would give some memory to it ini_set("memory_limit","64M"); my input file size is 1 GB. Is that memory limit is based on RAM we have on... (1 Reply)
Discussion started by: elamurugu
1 Replies

7. Programming

Copying 1024 bytes data in 3-bytes chunk

Hi, If I want to copy a 1024 byte data stream in to the target location in 3-bytes chunk, I guess I can use the following script. dd bs=1024 count=3 if=/src of=/dest But, I would like to know, how to do it via a C program. I have tried this with memcpy(), that did not help. (3 Replies)
Discussion started by: royalibrahim
3 Replies

8. UNIX for Dummies Questions & Answers

X bytes of 0, Y bytes of random data, Z bytes of 5, T bytes of 1. ??

Hello guys. I really hope someone will help me with this one.. So, I have to write this script who: - creates a file home/student/vmdisk of 10 mb - formats that file to ext3 - mounts that partition to /mnt/partition - creates a file /mnt/partition/data. In this file, there will... (1 Reply)
Discussion started by: razolo13
1 Replies

9. Shell Programming and Scripting

Shell script - entered input(1-40 bytes) needs to be converted exactly 40 bytes

hello, suppose, entered input is of 1-40 bytes, i need it to be converted to 40 bytes exactly. example: if i have entered my name anywhere between 1-40 i want it to be stored with 40 bytes exactly. enter your name: donald duck (this is of 11 bytes) expected is as below - display 11... (3 Replies)
Discussion started by: shravan.300
3 Replies

10. Shell Programming and Scripting

Get file's first x bytes

is there a better way to do this: head -c 10000k /var/dump.log | head -c 6000k unfortunately, the "-c" option is not available on sun solaris. so i'm looking at "dd". but i dont know how to use it to achieve the same exact goal as the above head command. this needs to work on both solaris... (5 Replies)
Discussion started by: SkySmart
5 Replies
pack(1) 							   User Commands							   pack(1)

NAME
pack, pcat, unpack - compress and expand files SYNOPSIS
pack [-f] [-] file... pcat file... unpack file... DESCRIPTION
pack The pack command attempts to store the specified files in a compressed form. Wherever possible (and useful), each input file file is replaced by a packed file file.z with the same access modes, access and modified dates, and owner as those of file. If pack is successful, file will be removed. The amount of compression obtained depends on the size of the input file and the character frequency distribution. Because a decoding tree forms the first part of each .z file, it is usually not worthwhile to pack files smaller than three blocks, unless the character frequency distribution is very skewed, which may occur with printer plots or pictures. Typically, text files are reduced to 60-75% of their original size. Load modules, which use a larger character set and have a more uniform distribution of characters, show little compression, the packed versions being about 90% of the original size. pack returns a value that is the number of files that it failed to compress. No packing will occur if: o the file appears to be already packed o the file name has more than 14 - 2 bytes o the file has links o the file is a directory o the file cannot be opened o the file is empty o no disk storage blocks will be saved by packing o a file called file.z already exists o the .z file cannot be created o an I/O error occurred during processing. The last segment of the file name must contain no more than 14 - 2 bytes to allow space for the appended .z extension. Directories cannot be compressed. pcat The pcat command does for packed files what cat(1) does for ordinary files, except that pcat cannot be used as a filter. The specified files are unpacked and written to the standard output. pcat returns the number of files it was unable to unpack. Failure may occur if: o the file cannot be opened; o the file does not appear to be the output of pack. unpack The unpack command expands files created by pack. For each file specified in the command, a search is made for a file called file.z (or just file, if file ends in .z). If this file appears to be a packed file, it is replaced by its expanded version. The new file has the .z suffix stripped from its name, and has the same access modes, access and modification dates, and owner as those of the packed file. unpack returns a value that is the number of files it was unable to unpack. Failure may occur for the same reasons that it may in pcat, as well as for the following: o a file with the ``unpacked'' name already exists; o the unpacked file cannot be created. o the filename (excluding the .z extension) has more than 14 bytes. OPTIONS
The following options are supported by pack: -f Forces packing of file. This is useful for causing an entire directory to be packed even if some of the files will not benefit. Packed files can be restored to their original form using unpack or pcat. OPERANDS
The following operands are supported: file A path name of a file to be packed, unpacked, or pcated; file can include or omit the .z suffix. - pack uses Huffman (minimum redundancy) codes on a byte-by-byte basis. If the - argument is used, an internal flag is set that causes the number of times each byte is used, its relative frequency, and the code for the byte to be printed on the standard output. Additional occurrences of - in place of file will cause the internal flag to be set and reset. USAGE
See largefile(5) for the description of the behavior of pack, pcat, and unpack when encountering files greater than or equal to 2 Gbyte ( 2 **31 bytes). EXAMPLES
Example 1: Viewing a Packed File To view a packed file named file.z use: example% pcat file.z or just: example% pcat file Example 2: Making and Unpacked Copy: To make an unpacked copy, say nnn, of a packed file named file.z (without destroying file.z) use the command: example% pcat file >nnn ENVIRONMENT VARIABLES
See environ(5) for descriptions of the following environment variables that affect the execution of pack, pcat, and unpack: LC_CTYPE, LC_MESSAGES, and NLSPATH. EXIT STATUS
The following exit values are returned: 0 Successful completion. >0 An error occurred. The number of files the command failed to pack/unpack is returned. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWesu | +-----------------------------+-----------------------------+ |CSI |Enabled | +-----------------------------+-----------------------------+ SEE ALSO
cat(1), compress(1), zcat(1), attributes(5), environ(5), largefile( 5) SunOS 5.10 20 Dec 1996 pack(1)
All times are GMT -4. The time now is 06:37 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy