06-26-2007
GZIP memory constraints
Currently I am using the ZLIB_VERSION "1.2.3" . The memory requirement for Zlib/GZIP compression is stated as
/* The memory requirements for deflate are (in bytes):
(1 << (windowBits+2)) + (1 << (memLevel+9))
that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values)
plus a few kilobytes for small objects. For example, if you want to reduce
the default memory requirements from 256K to 128K, compile with
make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7"
Of course this will generally degrade compression */
If I can make changes to these macro to reduce memory consumption,
MAX_WBITS=14
MAX_MEM_LEVEL = 7
How it will effect the GZIP compression method ?
What would be the memory requirement for window size, the size of Heads of the hash chains,
and prev pointer which is basically linking to older string with the same hash index?
Also, what are the various ways to reduce memory usage?
9 More Discussions You Might Find Interesting
1. Programming
Hi, i have class template,
1)can i override the copy constructor
2)can we have virtual function in class template
if not plz tel why? I tried , compile error comes for me...
Thanks
Sarwan (0 Replies)
Discussion started by: sarwan
0 Replies
2. Programming
I wrote a function which for uncompressing data for gzip or deflate format using zlib,see followed code;
source param is pointed to the compressed data,len param is the size of compressed data,
dest param is for returning the address which pointed to the uncompressed data;the last gzip param tell... (0 Replies)
Discussion started by: iwishfine
0 Replies
3. UNIX for Advanced & Expert Users
I am using the ZLIB_VERSION "1.2.3" . The memory requirement for Zlib/GZIP compression is stated as
/* The memory requirements for deflate are (in bytes):
(1 << (windowBits+2)) + (1 << (memLevel+9))
that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values)
... (0 Replies)
Discussion started by: Parmod Garg
0 Replies
4. UNIX for Advanced & Expert Users
Hi All,
I have a random test file: test.txt, size: 146
$ ll test.txt
$ 146 test.txt
Take 1:
$ cat test.txt | gzip > test.txt.gz
$ ll test.txt.gz
$ 124 test.txt.gz
Take 2:
$ gzip test.txt
$ ll test.txt.gz
$ 133 test.txt.gz
As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies
5. UNIX for Advanced & Expert Users
HI Team ,
I have interesting issue observed when using teradata sql assistant(14.1) and Informatica tool (9.5) versions.
I created SQL code in teradata where source count is 5000 records . I am using source and target database as teradata and trying to load using informatica tool . Its straight... (0 Replies)
Discussion started by: Perlbaby
0 Replies
6. Solaris
I have an x86 Solaris box running 11.2 and have run into the following issue when attempting to run a package update. Has anyone else come across this issue and resolved it successfully, or am I waiting on Oracle to release other updated packages?
uname -a
SunOS <hostname> 5.11 11.2... (13 Replies)
Discussion started by: nova_cyclist
13 Replies
7. Shell Programming and Scripting
I have a list file1 like
dog
cow
fox
cat
fish
duck
crowI want to classify the elements of file1 based on constrains applied on file2. Additionally the number of elements (words) in the each line of file2 is not fixed. This is my file2
cow cat fox dog
cow fox dog
fish crow fox dog cat ... (5 Replies)
Discussion started by: sammy777
5 Replies
8. UNIX for Dummies Questions & Answers
Hello Friends -
I am trying to grep certain messages that have a time slot like this:
MyRate=33FC|SystemDEF=445DE|Calc=33W2|Time=15:50:24
I am trying to grep everything after Time=15:50:26 including SystemDEF=E2S and Calc=33W2 into a file called myrate.dat
Not able to... (7 Replies)
Discussion started by: DallasT
7 Replies
9. Solaris
Hello Friends
I was running Solaris 11.3 x86.
Below is my configuration to set password Constraints.
more /etc/default/passwd
MAXWEEKS=4
MINWEEKS=1
WARNWEEKS=1
PASSLENGTH=8
NAMECHECK=YES
HISTORY=4
MINDIFF=3
MINALPHA=2 (4 Replies)
Discussion started by: jebby123
4 Replies
ZLIB(3) Library Functions Manual ZLIB(3)
NAME
zlib - compression/decompression library
SYNOPSIS
[see zlib.h for full description]
DESCRIPTION
The zlib library is a general purpose data compression library. The code is thread safe. It provides in-memory compression and decompres-
sion functions, including integrity checks of the uncompressed data. This version of the library supports only one compression method
(deflation) but other algorithms will be added later and will have the same stream interface.
Compression can be done in a single step if the buffers are large enough (for example if an input file is mmap'ed), or can be done by
repeated calls of the compression function. In the latter case, the application must provide more input and/or consume the output (provid-
ing more output space) before each call.
The library also supports reading and writing files in gzip (.gz) format with an interface similar to that of stdio.
The library does not install any signal handler. The decoder checks the consistency of the compressed data, so the library should never
crash even in case of corrupted input.
All functions of the compression library are documented in the file zlib.h. The distribution source includes examples of use of the
library the files example.c and minigzip.c.
A Java implementation of zlib is available in the Java Development Kit 1.1
http://www.javasoft.com/products/JDK/1.1/docs/api/Package-java.util.zip.html
A Perl interface to zlib, written by Paul Marquess (pmarquess@bfsec.bt.co.uk) is available at CPAN (Comprehensive Perl Archive Network)
sites, such as:
ftp://ftp.cis.ufl.edu/pub/perl/CPAN/modules/by-module/Compress/Compress-Zlib*
A Python interface to zlib written by A.M. Kuchling <amk@magnet.com> is available from the Python Software Association sites, such as:
ftp://ftp.python.org/pub/python/contrib/Encoding/zlib*.tar.gz
SEE ALSO
Questions about zlib should be sent to:
zlib@quest.jpl.nasa.gov or, if this fails, to the author addresses given below. The zlib home page is:
http://www.cdrom.com/pub/infozip/zlib/
The data format used by the zlib library is described by RFC (Request for Comments) 1950 to 1952 in the files:
ftp://ds.internic.net/rfc/rfc1950.txt (zlib format)
rfc1951.txt (deflate format)
rfc1952.txt (gzip format)
These documents are also available in other formats from:
ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html
AUTHORS
Version 1.1.4 Copyright (C) 1995-2002 Jean-loup Gailly (jloup@gzip.org) and Mark Adler (madler@alumni.caltech.edu).
This software is provided "as-is," without any express or implied warranty. In no event will the authors be held liable for any damages
arising from the use of this software. See the distribution directory with respect to requirements governing redistribution. The deflate
format used by zlib was defined by Phil Katz. The deflate and zlib specifications were written by L. Peter Deutsch. Thanks to all the
people who reported problems and suggested various improvements in zlib; who are too numerous to cite here.
UNIX manual page by R. P. C. Rodgers, U.S. National Library of Medicine (rodgers@nlm.nih.gov).
11 March 2002 ZLIB(3)