06-03-2011
gzip vs pipe gzip: produce different file size
Hi All,
I have a random test file: test.txt, size: 146
$ ll test.txt
$ 146 test.txt
Take 1:
$ cat test.txt | gzip > test.txt.gz
$ ll test.txt.gz
$ 124 test.txt.gz
Take 2:
$ gzip test.txt
$ ll test.txt.gz
$ 133 test.txt.gz
As you can see, gzipping a file and piping into gzip produce different sized *gz files. The problem currently is that some archivers, such as WinZip and Cygwin's gunzip have to unzip the piped gzip file twice before it can be read as a plain text file. Why is this and how can I ensure piping into gzip produces the same file as gzipping a file?
Thanks,
10 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
Hi,
How do I send a file as an attachment (gzip file) on a Unix system ? Using sendmail.
Please help me.
:confused: (3 Replies)
Discussion started by: lacca
3 Replies
2. Solaris
Hi experts,
I have thousands of file (data file and Gziped file) in same directory like below--
bash-2.05$ pwd
/home/mmc
bash-2.05$ file PP023149200709270546
TT023149200709270546: gzip compressed data - deflate method
bash-2.05$ file PP027443200711242320
TT027443200711242320: ... (10 Replies)
Discussion started by: thepurple
10 Replies
3. Solaris
I want to gzip a file and append the creation date to the end of the file. How can I accomplish this task. Basically they are log files which need a creation date stamp appended to make sure they do not overwrite other log files.
-jack (3 Replies)
Discussion started by: jacktravine
3 Replies
4. Shell Programming and Scripting
Guys,
I just want to know the difference in following (core difference)
zcat a.gz b.gz c.gz |gzip >d.gz
And
zcat a.gz >>d.gz
zcat b.gz >>d.gz
zcat c.gz >>d.gz
do we have 3 gzip header in 1st and only one in second case.
please let me know this in detail
Thanks,... (2 Replies)
Discussion started by: mohan_xunil
2 Replies
5. Programming
Hi all...
I have the following code:
FILE *fp = popen(" gzip -dc /somemount/somefile.gz", "r");
while(fgets(buffer, 1024, fp))
{
some code....
}
"/somemount" is a mount of some network drive. Sometimes error occurs in while loop - I can see the following "Input/Output error"... (4 Replies)
Discussion started by: adm1n
4 Replies
6. UNIX for Dummies Questions & Answers
I want gzip of a file in the same folder where it is kept now
$filename = '/var/dev/test.txt'
/opt/home/>> gzip -c $filename > test.txt.gz
however command creates it in the folder in /opt/home/
How to gzip a file in the same directory where it is now , no matter from where we execute and also... (2 Replies)
Discussion started by: lalitpct
2 Replies
7. Shell Programming and Scripting
Dear community,
I've a continuos tcpdump on redhat that close the dumped file every 100000 captured packets.
To avoid disk full I would like to gzip the closed *.cap file. But how can I check if the file is currently opened by tcpdump and skip it from gzip?
Thanks!
EDIT: Just to post an... (9 Replies)
Discussion started by: Lord Spectre
9 Replies
8. Shell Programming and Scripting
Hi,
Below is the description of my problem.
I am trying to loop through the below file names sitting in the file - FileNames.txt, and trying to decompress them
FileNames.txt
20150408_MGAC_.txt.gz
20150408_MGCC_.txt.gz
20150408_MGSH__.txt.gz
20150408_MGSL__.txt.gz
... (6 Replies)
Discussion started by: Richmond_CS
6 Replies
9. Shell Programming and Scripting
Hi all,
I am calling Temp.sh and it is has simple line
$gpath=`which gzip`
$gpath $FilePath/My_temp.log
if I run this script, logging to server then its works fine. But when I send this script over the SSH it does not work at it. gzip is command is not execute.
I am using
gzip 1.6... (2 Replies)
Discussion started by: girijajoshi
2 Replies
10. UNIX for Beginners Questions & Answers
Hello,
I am trying to send a gzip file on email using below command but the zipped file received on email is corrupt.
mailsend -smtp $smtpip -content-type 'application/x-gzip' -mime-type "application/x-gzip" -t $receiver -f $sender -sub "$subject" -M "$MSG" -attach $file
file name is ... (1 Reply)
Discussion started by: tushar.modgil
1 Replies
LEARN ABOUT MOJAVE
perlio::gzip
gzip(3) User Contributed Perl Documentation gzip(3)
NAME
PerlIO::gzip - Perl extension to provide a PerlIO layer to gzip/gunzip
SYNOPSIS
use PerlIO::gzip;
open FOO, "<:gzip", "file.gz" or die $!;
print while <FOO>; # And it will be uncompressed...
binmode FOO, ":gzip(none)" # Starts reading deflate stream from here on
DESCRIPTION
PerlIO::gzip provides a PerlIO layer that manipulates files in the format used by the "gzip" program. Compression and Decompression are
implemented, but not together. If you attempt to open a file for reading and writing the open will fail.
EXPORT
PerlIO::gzip exports no subroutines or symbols, just a perl layer "gzip"
LAYER ARGUMENTS
The "gzip" layer takes a comma separated list of arguments. 4 exclusive options choose the header checking mode:
gzip
The default. Expects a standard gzip file header for reading, writes a standard gzip file header.
none
Expects or writes no file header; assumes the file handle is immediately a deflate stream (eg as would be found inside a "zip" file)
auto
Potentially dangerous. If the first two bytes match the "gzip" header "x1fx8b" then a gzip header is assumed (and checked) else a
deflate stream is assumed. No different from gzip on writing.
autopop
Potentially dangerous. If the first two bytes match the "gzip" header "x1fx8b" then a gzip header is assumed (and checked) else the
layer is silently popped. This results in gzip files being transparently decompressed, other files being treated normally. Of course,
this has sides effects such as File::Copy becoming gunzip, and File::Compare comparing the uncompressed contents of files.
In autopop mode Opening a handle for writing (or reading and writing) will cause the gzip layer to automatically be popped.
Optionally you can add this flag:
lazy
For reading, defer header checking until the first read. For writing, don't write a header until the first buffer empty of compressed
data to disk. (and don't write anything at all if no data was written to the handle)
By default, gzip header checking is done before the "open" (or "binmode") returns, so if an error is detected in the gzip header the
"open" or "binmode" will fail. However, this will require reading some data, or writing a header. With lazy set on a file opened for
reading the check is deferred until the first read so the "open" should always succeed, but any problems with the header will cause an
error on read.
open FOO, "<:gzip(lazy)", "file.gz" or die $!; # Dangerous.
while (<FOO>) {
print;
} # Whoa. Bad. You're not distinguishing between errors and EOF.
If you're not careful you won't spot the errors - like the example above you'll think you got end of file.
lazy is ignored if you are in autopop mode.
AUTHOR
Nicholas Clark, <nwc10+perlio-gzip@colon.colondot.net>
SEE ALSO
perl, gzip, rfc 1952 <http://www.ietf.org/rfc/rfc1952.txt> (the gzip file format specification), rfc 1951
<http://www.ietf.org/rfc/rfc1951.txt> (DEFLATE compressed data format specification)
perl v5.18.2 2006-10-01 gzip(3)