How to reduce GZIP memory usage


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users How to reduce GZIP memory usage
# 1  
Old 06-26-2007
How to reduce GZIP memory usage

I am using the ZLIB_VERSION "1.2.3" . The memory requirement for Zlib/GZIP compression is stated as


/* The memory requirements for deflate are (in bytes):
(1 << (windowBits+2)) + (1 << (memLevel+9))
that is: 128K for windowBits=15 + 128K for memLevel = 8 (default values)
plus a few kilobytes for small objects. For example, if you want to reduce
the default memory requirements from 256K to 128K, compile with
make CFLAGS="-O -DMAX_WBITS=14 -DMAX_MEM_LEVEL=7"
Of course this will generally degrade compression */

If I can make changes to these macro to reduce memory consumption,
MAX_WBITS=14
MAX_MEM_LEVEL = 7

How it will effect the GZIP compression method ?
Now what would be the memory requirement for window size, the size of Heads of the hash chains
and prev pointer which is basically linking to older string with the same hash index?

Also, what are the various ways to reduce memory usage?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Does it make sense to reduce the total shared memory

We have several dozen Redhat 5, 6 and 7 servers that are running Oracle databases. On some databases we are using automatic memory management, which uses shared memory. On other databases we are use manual memory management, which does not use shared memory. When I see that a server is swapping... (2 Replies)
Discussion started by: gandolf989
2 Replies

2. UNIX for Dummies Questions & Answers

Memory usage per user,percent usage,sytem time in ksh

Let's say i have 20 users logged on Server. How can I know how much memory percent used each of them is using with system time in each user? (2 Replies)
Discussion started by: roy1912
2 Replies

3. AIX

How to reduce and findout CPU usage?

Frequently i am getting "total CPU is now 100.00%, which is above the error threshold (90%)" this error . (3 Replies)
Discussion started by: ranjithm
3 Replies

4. UNIX for Dummies Questions & Answers

Command to display the space usage (memory usage) of a specific directory.

Hi all, Can you please tell me the command, with which one can know the amount of space a specific directory has used. df -k . ---> Displays, the amount of space allocated, and used for a directory. du -k <dir name> - gives me the memory used of all the files inside <dir> But i... (2 Replies)
Discussion started by: abhisheksunkari
2 Replies

5. AIX

How to monitor the IBM AIX server for I/O usage,memory usage,CPU usage,network..?

How to monitor the IBM AIX server for I/O usage, memory usage, CPU usage, network usage, storage usage? (3 Replies)
Discussion started by: laknar
3 Replies

6. Solaris

current CPU usage, memory usage, disk I/O oid(snmp)

Hi, I want to monitor the current cpu usage, monitor usage , disk I/o and network utlization for solaris using SNMP. I want the oids for above tasks. can you please tell me that Thank you (2 Replies)
Discussion started by: S_venkatesh
2 Replies

7. Linux

Using SQUID to reduce traffic usage in the office - how effective?

We are working in the office where about 5-6 machines have Internet access. We pay for each GB of traffic we consume and that's quite expensive. Almost no worker download files. Just surfing websites (including our corporate one that is located outside of local network thus we pay for accessing it... (5 Replies)
Discussion started by: FractalizeR
5 Replies

8. HP-UX

how can I find cpu usage memory usage swap usage and logical volume usage

how can I find cpu usage memory usage swap usage and I want to know CPU usage above X% and contiue Y times and memory usage above X % and contiue Y times my final destination is monitor process logical volume usage above X % and number of Logical voluage above can I not to... (3 Replies)
Discussion started by: alert0919
3 Replies

9. HP-UX

How can I get memory usage or anything that show memory used from sar file?

Refer from title: How can i get memory used or anything that can show memory from sar file example on solaris:- we can use sar with option to show memory used at time that sar crontab run. on HP-UX, it not has option to see memory used. But i think it may be have some parameter or some... (1 Reply)
Discussion started by: panithat
1 Replies

10. Programming

Monitor CPU usage and Memory Usage

how can i monitor usages of CPU, Memory, Hard disk etc. under SUN Solaries through a c program or java program i want to store that data into database so i can show it graphically thanks in advance (2 Replies)
Discussion started by: Gajanad Bihani
2 Replies
Login or Register to Ask a Question
IO::Zlib(3pm)						 Perl Programmers Reference Guide					     IO::Zlib(3pm)

NAME
IO::Zlib - IO:: style interface to Compress::Zlib SYNOPSIS
With any version of Perl 5 you can use the basic OO interface: use IO::Zlib; $fh = new IO::Zlib; if ($fh->open("file.gz", "rb")) { print <$fh>; $fh->close; } $fh = IO::Zlib->new("file.gz", "wb9"); if (defined $fh) { print $fh "bar "; $fh->close; } $fh = IO::Zlib->new("file.gz", "rb"); if (defined $fh) { print <$fh>; undef $fh; # automatically closes the file } With Perl 5.004 you can also use the TIEHANDLE interface to access compressed files just like ordinary files: use IO::Zlib; tie *FILE, 'IO::Zlib', "file.gz", "wb"; print FILE "line 1 line2 "; tie *FILE, 'IO::Zlib', "file.gz", "rb"; while (<FILE>) { print "LINE: ", $_ }; DESCRIPTION
"IO::Zlib" provides an IO:: style interface to Compress::Zlib and hence to gzip/zlib compressed files. It provides many of the same methods as the IO::Handle interface. Starting from IO::Zlib version 1.02, IO::Zlib can also use an external gzip command. The default behaviour is to try to use an external gzip if no "Compress::Zlib" can be loaded, unless explicitly disabled by use IO::Zlib qw(:gzip_external 0); If explicitly enabled by use IO::Zlib qw(:gzip_external 1); then the external gzip is used instead of "Compress::Zlib". CONSTRUCTOR
new ( [ARGS] ) Creates an "IO::Zlib" object. If it receives any parameters, they are passed to the method "open"; if the open fails, the object is destroyed. Otherwise, it is returned to the caller. OBJECT METHODS
open ( FILENAME, MODE ) "open" takes two arguments. The first is the name of the file to open and the second is the open mode. The mode can be anything acceptable to Compress::Zlib and by extension anything acceptable to zlib (that basically means POSIX fopen() style mode strings plus an optional number to indicate the compression level). opened Returns true if the object currently refers to a opened file. close Close the file associated with the object and disassociate the file from the handle. Done automatically on destroy. getc Return the next character from the file, or undef if none remain. getline Return the next line from the file, or undef on end of string. Can safely be called in an array context. Currently ignores $/ ($INPUT_RECORD_SEPARATOR or $RS when English is in use) and treats lines as delimited by " ". getlines Get all remaining lines from the file. It will croak() if accidentally called in a scalar context. print ( ARGS... ) Print ARGS to the file. read ( BUF, NBYTES, [OFFSET] ) Read some bytes from the file. Returns the number of bytes actually read, 0 on end-of-file, undef on error. eof Returns true if the handle is currently positioned at end of file? seek ( OFFSET, WHENCE ) Seek to a given position in the stream. Not yet supported. tell Return the current position in the stream, as a numeric offset. Not yet supported. setpos ( POS ) Set the current position, using the opaque value returned by "getpos()". Not yet supported. getpos ( POS ) Return the current position in the string, as an opaque object. Not yet supported. USING THE EXTERNAL GZIP
If the external gzip is used, the following "open"s are used: open(FH, "gzip -dc $filename |") # for read opens open(FH, " | gzip > $filename") # for write opens You can modify the 'commands' for example to hardwire an absolute path by e.g. use IO::Zlib ':gzip_read_open' => '/some/where/gunzip -c %s |'; use IO::Zlib ':gzip_write_open' => '| /some/where/gzip.exe > %s'; The %s is expanded to be the filename ("sprintf" is used, so be careful to escape any other "%" signs). The 'commands' are checked for sanity - they must contain the %s, and the read open must end with the pipe sign, and the write open must begin with the pipe sign. CLASS METHODS
has_Compress_Zlib Returns true if "Compress::Zlib" is available. Note that this does not mean that "Compress::Zlib" is being used: see "gzip_external" and gzip_used. gzip_external Undef if an external gzip can be used if "Compress::Zlib" is not available (see "has_Compress_Zlib"), true if an external gzip is explicitly used, false if an external gzip must not be used. See "gzip_used". gzip_used True if an external gzip is being used, false if not. gzip_read_open Return the 'command' being used for opening a file for reading using an external gzip. gzip_write_open Return the 'command' being used for opening a file for writing using an external gzip. DIAGNOSTICS
IO::Zlib::getlines: must be called in list context If you want read lines, you must read in list context. IO::Zlib::gzopen_external: mode '...' is illegal Use only modes 'rb' or 'wb' or /wb[1-9]/. IO::Zlib::import: '...' is illegal The known import symbols are the ":gzip_external", ":gzip_read_open", and ":gzip_write_open". Anything else is not recognized. IO::Zlib::import: ':gzip_external' requires an argument The ":gzip_external" requires one boolean argument. IO::Zlib::import: 'gzip_read_open' requires an argument The ":gzip_external" requires one string argument. IO::Zlib::import: 'gzip_read' '...' is illegal The ":gzip_read_open" argument must end with the pipe sign (|) and have the %s for the filename. See "USING THE EXTERNAL GZIP". IO::Zlib::import: 'gzip_write_open' requires an argument The ":gzip_external" requires one string argument. IO::Zlib::import: 'gzip_write_open' '...' is illegal The ":gzip_write_open" argument must begin with the pipe sign (|) and have the %s for the filename. An output redirect (>) is also often a good idea, depending on your operating system shell syntax. See "USING THE EXTERNAL GZIP". IO::Zlib::import: no Compress::Zlib and no external gzip Given that we failed to load "Compress::Zlib" and that the use of an external gzip was disabled, IO::Zlib has not much chance of working. IO::Zlib::open: needs a filename No filename, no open. IO::Zlib::READ: NBYTES must be specified We must know how much to read. IO::Zlib::WRITE: too long LENGTH The LENGTH must be less than or equal to the buffer size. SEE ALSO
perlfunc, "I/O Operators" in perlop, IO::Handle, Compress::Zlib HISTORY
Created by Tom Hughes <tom@compton.nu>. Support for external gzip added by Jarkko Hietaniemi <jhi@iki.fi>. COPYRIGHT
Copyright (c) 1998-2004 Tom Hughes <tom@compton.nu>. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.18.2 2013-11-04 IO::Zlib(3pm)