Sponsored Content
Full Discussion: General compression question
Operating Systems Linux General compression question Post 302524042 by Perderabo on Friday 20th of May 2011 09:08:49 PM
Old 05-20-2011
Data compression requires a lot of CPU power. To compress several hundred files simultaneously you're simply going to need a lot of horsepower. Something like a 16 core server with 64 GB of memory is probably the minimum configuration for this task. bzip2 is best compression program I know. There are a few alternatives but they are not as good. But there no magic software that can compress hundreds of 4 GB files on a workstation in a twinkling of an eye.
 

10 More Discussions You Might Find Interesting

1. IP Networking

General Proxy Question

Firstly I must apologise; I have posted something similar in the "general UNIX newbies" forum; but in reterospect I think that it's more appropriate here. Anyhoo; I am trying to detect whether people accessing one of my servers are coming via a proxied connection. I must add that this is... (1 Reply)
Discussion started by: sam_pointer
1 Replies

2. UNIX for Dummies Questions & Answers

General Proxy Question

This is quite a general question: I am trying to detect whether people accessing my network are using a proxy server. This is *not* to ensure that web pages are not cached! The only way that I can think of doing this is to intercept at packet level and examine the source port for... (1 Reply)
Discussion started by: sam_pointer
1 Replies

3. UNIX for Dummies Questions & Answers

General Question

Hi, I've been racking my brains trying to remember, but, whats the command to change the default shell? I'm currently always in the Korn shell and I want to start out in the Bash shell. I'm running a variant of BSD I guess in Mac OS X 10.2.2 and Mandrake. Thanks. ccindyderek:confused: (4 Replies)
Discussion started by: ccindyderek
4 Replies

4. Shell Programming and Scripting

general question?

Perl, Python, and PHP are these languages easy to use? Are they command line or are they part of a GUI? (2 Replies)
Discussion started by: wmosley2
2 Replies

5. Shell Programming and Scripting

general question on executing file in drwx--x--x directory

suppose: nevermind, i figured it out, haha... (2 Replies)
Discussion started by: cul8erppl
2 Replies

6. UNIX for Dummies Questions & Answers

Executable directory general question

This may be a dumb question (but this is the UNIX for Dummies Q&A forum :) But I'm wondering, what is the purpose of a directory being given execute permissions? I can't execute a directory, only files, right? If I make a directory executable, the files inside still won't execute unless I give... (1 Reply)
Discussion started by: FredSmith
1 Replies

7. UNIX for Dummies Questions & Answers

General Programming Question

Experience level : New to programming in Linux. Forgive my noobiness in this context with regards to programming, language or grammar. Some Background info : I have seen a lot of programs which I use (Modo and Maya especially) which can use a command line to run the different parts of the... (2 Replies)
Discussion started by: snd321
2 Replies

8. Solaris

General question about configuring DHCP on Solaris

Dears, my question is that possible to configure any server to work in solaris zone i mean any server cos i read that DHCP can't be work on solaris zone is that real or not Best Regards Alexander Corvinus (4 Replies)
Discussion started by: corvinusbsd
4 Replies

9. UNIX for Dummies Questions & Answers

General question about folder level permissions

How is the level of access on a particular folder determined? I have heard (its just hearsay so am not particularly sure of it) that the access a particular user/group has to a low level directory is also affected by the level of access granted to the user/group on its parent directories. e.g. ... (1 Reply)
Discussion started by: jawsnnn
1 Replies

10. Programming

General question about learning Python

I am planning on taking a class in Python. My choices are 2.5 or 3.0. Which version should I choose? I am getting the impression they are two separate paths. thanks. (5 Replies)
Discussion started by: djehresmann
5 Replies
COMPRESS(1)						    BSD General Commands Manual 					       COMPRESS(1)

NAME
compress, uncompress, -- compress and expand data SYNOPSIS
compress [-cfv] [-b bits] [file ...] uncompress [-cfv] [file ...] DESCRIPTION
The compress utility reduces the size of the named files using adaptive Lempel-Ziv coding. Each file is renamed to the same name plus the extension ``.Z''. As many of the modification time, access time, file flags, file mode, user ID, and group ID as allowed by permissions are retained in the new file. If compression would not reduce the size of a file, the file is ignored. The uncompress utility restores the compressed files to their original form, renaming the files by deleting the ``.Z'' extension. If renaming the files would cause files to be overwritten and the standard input device is a terminal, the user is prompted (on the standard error output) for confirmation. If prompting is not possible or confirmation is not received, the files are not overwritten. If no files are specified or a file argument is a single dash ('-'), the standard input is compressed or uncompressed to the standard output. If either the input and output files are not regular files, the checks for reduction in size and file overwriting are not performed, the input file is not removed, and the attributes of the input file are not retained. The options are as follows: -b Specify the bits code limit (see below). -c Compressed or uncompressed output is written to the standard output. No files are modified. -f Force compression of file, even if it is not actually reduced in size. Additionally, files are overwritten without prompting for confirmation. -v Print the percentage reduction of each file. The compress utility uses a modified Lempel-Ziv algorithm. Common substrings in the file are first replaced by 9-bit codes 257 and up. When code 512 is reached, the algorithm switches to 10-bit codes and continues to use more bits until the limit specified by the -b flag is reached (the default is 16). Bits must be between 9 and 16. After the bits limit is reached, compress periodically checks the compression ratio. If it is increasing, compress continues to use the existing code dictionary. However, if the compression ratio decreases, compress discards the table of substrings and rebuilds it from scratch. This allows the algorithm to adapt to the next "block" of the file. The -b flag is omitted for uncompress since the bits parameter specified during compression is encoded within the output, along with a magic number to ensure that neither decompression of random data nor recompression of compressed data is attempted. The amount of compression obtained depends on the size of the input, the number of bits per code, and the distribution of common substrings. Typically, text such as source code or English is reduced by 50-60%. Compression is generally much better than that achieved by Huffman cod- ing (as used in the historical command pack), or adaptive Huffman coding (as used in the historical command compact), and takes less time to compute. DIAGNOSTICS
The compress and uncompress utilities exit 0 on success, and >0 if an error occurs. The compress utility exits 2 if attempting to compress the file would not reduce its size and the -f option was not specified. SEE ALSO
gunzip(1), gzexe(1), gzip(1), zcat(1), zmore(1), znew(1) Welch, Terry A., "A Technique for High Performance Data Compression", IEEE Computer, 17:6, pp. 8-19, June, 1984. STANDARDS
The compress and uncompress utilities conform to IEEE Std 1003.1-2001 (``POSIX.1''). HISTORY
The compress command appeared in 4.3BSD. BSD
May 17, 2002 BSD
All times are GMT -4. The time now is 08:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy