Sponsored Content
Top Forums UNIX for Advanced & Expert Users gzip vs pipe gzip: produce different file size Post 302527627 by drl on Friday 3rd of June 2011 10:48:57 PM
Old 06-03-2011
Hi.

I did not need to unzip twice. Perhaps this will help explain the difference in lengths:
Code:
#!/usr/bin/env bash

# @(#) s1	Demonstrate gzip lengths.

# Utility functions: print-as-echo, print-line-with-visual-space, debug.
pe() { for i;do printf "%s" "$i";done; printf "\n"; }
pl() { pe;pe "-----" ;pe "$*"; }
db() { ( printf " db, ";for i;do printf "%s" "$i";done; printf "\n" ) >&2 ; }
db() { : ; }
C=$HOME/bin/context && [ -f $C ] && $C spit gzip

rm -f f1 f1.gz f2 f2.gz f3 f3.gz f4 f4.gz f5 f5.gz
spit -c 10 -r 5 > f1
cp f1 f2
cp f1 f3

pl " Input data files f1, f2:"
wc f1 f2

pl " Results, re-direction:"
cat f1 | gzip > f1.gz
wc f1.gz
rm -f f1

pl " Results, by filename:"
gzip -n f2
wc f2.gz

pl " Results, by filename, plain:"
gzip f3
wc f3.gz

cp f1.gz f4.gz
cp f3.gz f5.gz

pl " All files compressed, contents:"
ls -lgG f*
pe
file f*

pl " All files uncompressed:"
for f in *.gz
do
  gunzip $f
done
ls -lgG f*

exit 0

producing:
Code:
% ./s1

Environment: LC_ALL = C, LANG = C
(Versions displayed with local utility "version")
OS, ker|rel, machine: Linux, 2.6.26-2-amd64, x86_64
Distribution        : Debian GNU/Linux 5.0.8 (lenny) 
GNU bash 3.2.39
spit - ( local: RepRev 1.3, ~/bin/spit, 2011-04-29 )
gzip 1.3.12

-----
 Input data files f1, f2:
  5  50 100 f1
  5  50 100 f2
 10 100 200 total

-----
 Results, re-direction:
 0  1 30 f1.gz

-----
 Results, by filename:
 0  1 30 f2.gz

-----
 Results, by filename, plain:
 0  1 33 f3.gz

-----
 All files compressed, contents:
-rw-r--r-- 1 30 Jun  3 21:45 f1.gz
-rw-r--r-- 1 30 Jun  3 21:45 f2.gz
-rw-r--r-- 1 33 Jun  3 21:45 f3.gz
-rw-r--r-- 1 30 Jun  3 21:45 f4.gz
-rw-r--r-- 1 33 Jun  3 21:45 f5.gz

f1.gz: gzip compressed data, from Unix, last modified: Fri Jun  3 21:45:43 2011
f2.gz: gzip compressed data, from Unix
f3.gz: gzip compressed data, was "f3", from Unix, last modified: Fri Jun  3 21:45:43 2011
f4.gz: gzip compressed data, from Unix, last modified: Fri Jun  3 21:45:43 2011
f5.gz: gzip compressed data, was "f3", from Unix, last modified: Fri Jun  3 21:45:43 2011

-----
 All files uncompressed:
-rw-r--r-- 1 100 Jun  3 21:45 f1
-rw-r--r-- 1 100 Jun  3 21:45 f2
-rw-r--r-- 1 100 Jun  3 21:45 f3
-rw-r--r-- 1 100 Jun  3 21:45 f4
-rw-r--r-- 1 100 Jun  3 21:45 f5

See man gzip for details.

Best wishes ... cheers, drl
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How do I send a file as an attachment (gzip file) on a Unix system

Hi, How do I send a file as an attachment (gzip file) on a Unix system ? Using sendmail. Please help me. :confused: (3 Replies)
Discussion started by: lacca
3 Replies

2. Solaris

script for Gzip thousands of file

Hi experts, I have thousands of file (data file and Gziped file) in same directory like below-- bash-2.05$ pwd /home/mmc bash-2.05$ file PP023149200709270546 TT023149200709270546: gzip compressed data - deflate method bash-2.05$ file PP027443200711242320 TT027443200711242320: ... (10 Replies)
Discussion started by: thepurple
10 Replies

3. Solaris

gzip a file and append creation date stamp to file

I want to gzip a file and append the creation date to the end of the file. How can I accomplish this task. Basically they are log files which need a creation date stamp appended to make sure they do not overwrite other log files. -jack (3 Replies)
Discussion started by: jacktravine
3 Replies

4. Shell Programming and Scripting

Appending a GZIP File

Guys, I just want to know the difference in following (core difference) zcat a.gz b.gz c.gz |gzip >d.gz And zcat a.gz >>d.gz zcat b.gz >>d.gz zcat c.gz >>d.gz do we have 3 gzip header in 1st and only one in second case. please let me know this in detail Thanks,... (2 Replies)
Discussion started by: mohan_xunil
2 Replies

5. Programming

gzip on pipe error handling

Hi all... I have the following code: FILE *fp = popen(" gzip -dc /somemount/somefile.gz", "r"); while(fgets(buffer, 1024, fp)) { some code.... } "/somemount" is a mount of some network drive. Sometimes error occurs in while loop - I can see the following "Input/Output error"... (4 Replies)
Discussion started by: adm1n
4 Replies

6. UNIX for Dummies Questions & Answers

gzip of a file in the same folder

I want gzip of a file in the same folder where it is kept now $filename = '/var/dev/test.txt' /opt/home/>> gzip -c $filename > test.txt.gz however command creates it in the folder in /opt/home/ How to gzip a file in the same directory where it is now , no matter from where we execute and also... (2 Replies)
Discussion started by: lalitpct
2 Replies

7. Shell Programming and Scripting

Check if file is locked otherwise gzip it

Dear community, I've a continuos tcpdump on redhat that close the dumped file every 100000 captured packets. To avoid disk full I would like to gzip the closed *.cap file. But how can I check if the file is currently opened by tcpdump and skip it from gzip? Thanks! EDIT: Just to post an... (9 Replies)
Discussion started by: Lord Spectre
9 Replies

8. Shell Programming and Scripting

File name manipulation when extracting from gzip

Hi, Below is the description of my problem. I am trying to loop through the below file names sitting in the file - FileNames.txt, and trying to decompress them FileNames.txt 20150408_MGAC_.txt.gz 20150408_MGCC_.txt.gz 20150408_MGSH__.txt.gz 20150408_MGSL__.txt.gz ... (6 Replies)
Discussion started by: Richmond_CS
6 Replies

9. Shell Programming and Scripting

Shell script not able to zip the file using gzip

Hi all, I am calling Temp.sh and it is has simple line $gpath=`which gzip` $gpath $FilePath/My_temp.log if I run this script, logging to server then its works fine. But when I send this script over the SSH it does not work at it. gzip is command is not execute. I am using gzip 1.6... (2 Replies)
Discussion started by: girijajoshi
2 Replies

10. UNIX for Beginners Questions & Answers

MIME type for sending gzip file as attachment in email

Hello, I am trying to send a gzip file on email using below command but the zipped file received on email is corrupt. mailsend -smtp $smtpip -content-type 'application/x-gzip' -mime-type "application/x-gzip" -t $receiver -f $sender -sub "$subject" -M "$MSG" -attach $file file name is ... (1 Reply)
Discussion started by: tushar.modgil
1 Replies
gzip(3pm)						User Contributed Perl Documentation						 gzip(3pm)

NAME
PerlIO::gzip - Perl extension to provide a PerlIO layer to gzip/gunzip SYNOPSIS
use PerlIO::gzip; open FOO, "<:gzip", "file.gz" or die $!; print while <FOO>; # And it will be uncompressed... binmode FOO, ":gzip(none)" # Starts reading deflate stream from here on DESCRIPTION
PerlIO::gzip provides a PerlIO layer that manipulates files in the format used by the "gzip" program. Compression and Decompression are implemented, but not together. If you attempt to open a file for reading and writing the open will fail. EXPORT
PerlIO::gzip exports no subroutines or symbols, just a perl layer "gzip" LAYER ARGUMENTS
The "gzip" layer takes a comma separated list of arguments. 4 exclusive options choose the header checking mode: gzip The default. Expects a standard gzip file header for reading, writes a standard gzip file header. none Expects or writes no file header; assumes the file handle is immediately a deflate stream (eg as would be found inside a "zip" file) auto Potentially dangerous. If the first two bytes match the "gzip" header "x1fx8b" then a gzip header is assumed (and checked) else a deflate stream is assumed. No different from gzip on writing. autopop Potentially dangerous. If the first two bytes match the "gzip" header "x1fx8b" then a gzip header is assumed (and checked) else the layer is silently popped. This results in gzip files being transparently decompressed, other files being treated normally. Of course, this has sides effects such as File::Copy becoming gunzip, and File::Compare comparing the uncompressed contents of files. In autopop mode Opening a handle for writing (or reading and writing) will cause the gzip layer to automatically be popped. Optionally you can add this flag: lazy For reading, defer header checking until the first read. For writing, don't write a header until the first buffer empty of compressed data to disk. (and don't write anything at all if no data was written to the handle) By default, gzip header checking is done before the "open" (or "binmode") returns, so if an error is detected in the gzip header the "open" or "binmode" will fail. However, this will require reading some data, or writing a header. With lazy set on a file opened for reading the check is deferred until the first read so the "open" should always succeed, but any problems with the header will cause an error on read. open FOO, "<:gzip(lazy)", "file.gz" or die $!; # Dangerous. while (<FOO>) { print; } # Whoa. Bad. You're not distinguishing between errors and EOF. If you're not careful you won't spot the errors - like the example above you'll think you got end of file. lazy is ignored if you are in autopop mode. AUTHOR
Nicholas Clark, <nwc10+perlio-gzip@colon.colondot.net> SEE ALSO
perl, gzip, rfc 1952 <http://www.ietf.org/rfc/rfc1952.txt> (the gzip file format specification), rfc 1951 <http://www.ietf.org/rfc/rfc1951.txt> (DEFLATE compressed data format specification) perl v5.14.2 2006-10-01 gzip(3pm)
All times are GMT -4. The time now is 07:29 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy