Sponsored Content
Full Discussion: GZIP memory constraints
Top Forums UNIX for Advanced & Expert Users GZIP memory constraints Post 302123508 by Parmod Garg on Tuesday 26th of June 2007 01:59:22 AM
Old 06-26-2007
GZIP memory constraints

Currently I am using the ZLIB_VERSION "1.2.3" . The memory requirement for Zlib/GZIP compression is stated as


/* The memory requirements for deflate are (in bytes):
(1 << (windowBits+2)) + (1 << (memLevel+9))
that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values)
plus a few kilobytes for small objects. For example, if you want to reduce
the default memory requirements from 256K to 128K, compile with
make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7"
Of course this will generally degrade compression */

If I can make changes to these macro to reduce memory consumption,
MAX_WBITS=14
MAX_MEM_LEVEL = 7

How it will effect the GZIP compression method ?
What would be the memory requirement for window size, the size of Heads of the hash chains,
and prev pointer which is basically linking to older string with the same hash index?

Also, what are the various ways to reduce memory usage?
 

9 More Discussions You Might Find Interesting

1. Programming

About template constraints

Hi, i have class template, 1)can i override the copy constructor 2)can we have virtual function in class template if not plz tel why? I tried , compile error comes for me... Thanks Sarwan (0 Replies)
Discussion started by: sarwan
0 Replies

2. Programming

problem about using zlib to uncompress gzip in memory

I wrote a function which for uncompressing data for gzip or deflate format using zlib,see followed code; source param is pointed to the compressed data,len param is the size of compressed data, dest param is for returning the address which pointed to the uncompressed data;the last gzip param tell... (0 Replies)
Discussion started by: iwishfine
0 Replies

3. UNIX for Advanced & Expert Users

How to reduce GZIP memory usage

I am using the ZLIB_VERSION "1.2.3" . The memory requirement for Zlib/GZIP compression is stated as /* The memory requirements for deflate are (in bytes): (1 << (windowBits+2)) + (1 << (memLevel+9)) that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values) ... (0 Replies)
Discussion started by: Parmod Garg
0 Replies

4. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies

5. UNIX for Advanced & Expert Users

Teradata and Informatica Load constraints

HI Team , I have interesting issue observed when using teradata sql assistant(14.1) and Informatica tool (9.5) versions. I created SQL code in teradata where source count is 5000 records . I am using source and target database as teradata and trying to load using informatica tool . Its straight... (0 Replies)
Discussion started by: Perlbaby
0 Replies

6. Solaris

Pkg update: No solution was found to satisfy constraints

I have an x86 Solaris box running 11.2 and have run into the following issue when attempting to run a package update. Has anyone else come across this issue and resolved it successfully, or am I waiting on Oracle to release other updated packages? uname -a SunOS <hostname> 5.11 11.2... (13 Replies)
Discussion started by: nova_cyclist
13 Replies

7. Shell Programming and Scripting

Parsing a file based on positional constraints

I have a list file1 like dog cow fox cat fish duck crowI want to classify the elements of file1 based on constrains applied on file2. Additionally the number of elements (words) in the each line of file2 is not fixed. This is my file2 cow cat fox dog cow fox dog fish crow fox dog cat ... (5 Replies)
Discussion started by: sammy777
5 Replies

8. UNIX for Dummies Questions & Answers

Grep with time constraints

Hello Friends - I am trying to grep certain messages that have a time slot like this: MyRate=33FC|SystemDEF=445DE|Calc=33W2|Time=15:50:24 I am trying to grep everything after Time=15:50:26 including SystemDEF=E2S and Calc=33W2 into a file called myrate.dat Not able to... (7 Replies)
Discussion started by: DallasT
7 Replies

9. Solaris

Need help in configuring Password Constraints for Solaris 11.3

Hello Friends I was running Solaris 11.3 x86. Below is my configuration to set password Constraints. more /etc/default/passwd MAXWEEKS=4 MINWEEKS=1 WARNWEEKS=1 PASSLENGTH=8 NAMECHECK=YES HISTORY=4 MINDIFF=3 MINALPHA=2 (4 Replies)
Discussion started by: jebby123
4 Replies
ZLIB(3) 						     Library Functions Manual							   ZLIB(3)

NAME
zlib - compression/decompression library SYNOPSIS
[see zlib.h for full description] DESCRIPTION
The zlib library is a general purpose data compression library. The code is thread safe. It provides in-memory compression and decompres- sion functions, including integrity checks of the uncompressed data. This version of the library supports only one compression method (deflation) but other algorithms will be added later and will have the same stream interface. Compression can be done in a single step if the buffers are large enough (for example if an input file is mmap'ed), or can be done by repeated calls of the compression function. In the latter case, the application must provide more input and/or consume the output (provid- ing more output space) before each call. The library also supports reading and writing files in gzip (.gz) format with an interface similar to that of stdio. The library does not install any signal handler. The decoder checks the consistency of the compressed data, so the library should never crash even in case of corrupted input. All functions of the compression library are documented in the file zlib.h. The distribution source includes examples of use of the library the files example.c and minigzip.c. A Java implementation of zlib is available in the Java Development Kit 1.1 http://www.javasoft.com/products/JDK/1.1/docs/api/Package-java.util.zip.html A Perl interface to zlib, written by Paul Marquess (pmarquess@bfsec.bt.co.uk) is available at CPAN (Comprehensive Perl Archive Network) sites, such as: ftp://ftp.cis.ufl.edu/pub/perl/CPAN/modules/by-module/Compress/Compress-Zlib* A Python interface to zlib written by A.M. Kuchling <amk@magnet.com> is available from the Python Software Association sites, such as: ftp://ftp.python.org/pub/python/contrib/Encoding/zlib*.tar.gz SEE ALSO
Questions about zlib should be sent to: zlib@quest.jpl.nasa.gov or, if this fails, to the author addresses given below. The zlib home page is: http://www.cdrom.com/pub/infozip/zlib/ The data format used by the zlib library is described by RFC (Request for Comments) 1950 to 1952 in the files: ftp://ds.internic.net/rfc/rfc1950.txt (zlib format) rfc1951.txt (deflate format) rfc1952.txt (gzip format) These documents are also available in other formats from: ftp://ftp.uu.net/graphics/png/documents/zlib/zdoc-index.html AUTHORS
Version 1.1.4 Copyright (C) 1995-2002 Jean-loup Gailly (jloup@gzip.org) and Mark Adler (madler@alumni.caltech.edu). This software is provided "as-is," without any express or implied warranty. In no event will the authors be held liable for any damages arising from the use of this software. See the distribution directory with respect to requirements governing redistribution. The deflate format used by zlib was defined by Phil Katz. The deflate and zlib specifications were written by L. Peter Deutsch. Thanks to all the people who reported problems and suggested various improvements in zlib; who are too numerous to cite here. UNIX manual page by R. P. C. Rodgers, U.S. National Library of Medicine (rodgers@nlm.nih.gov). 11 March 2002 ZLIB(3)
All times are GMT -4. The time now is 07:15 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy