Show Percentage Compression in GZIP


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Show Percentage Compression in GZIP
# 1  
Old 07-16-2013
Lightbulb Show Percentage Compression in GZIP

Hi,

I used gzip command to compress a huge tar file. But I saw that compression % was more than 100%.
It might have inflated instead , probably because tar file is already packed properly.

So I thought of unzippping it. Now after unzip I expected the tar file to be of less size than .tar.gz file. But ,to my surprise it was more than that. Does that mean gzip actually reduced size eventhough , it show % compression more than 100%?

Here are statistics:

HTML Code:
after gzip
a.tar.gz  -  20,915,558,979
 
after gunzip
a.tar        - 22,213,027,840
 
Compression % = 175.7 
(Sorry, I forgot to check size of original tar , means tar before I zipped)

Your suggestions are greatly appreciated Smilie.

Thanks
# 2  
Old 07-16-2013
Quote:
Originally Posted by vinay4889
Hi,

I used gzip command to compress a huge tar file. But I saw that compression % was more than 100%.
It might have inflated instead , probably because tar file is already packed properly.
gzip won't ever make a file that much larger than it started, so it must be meaning the opposite of what you think.
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 07-16-2013
Quote:
Originally Posted by Corona688
gzip won't ever make a file that much larger than it started, so it must be meaning the opposite of what you think.
Thanks for quick resonse.
I am confused a bit. My assumption was based upon some pages on Internet. Can you please take a look at below.

Why GZip compression increases the file size for some extension?

Also , if it is really compressed why would it show compresion % more than 100%. I checked for some small files and compression % was 91%.
# 4  
Old 07-16-2013
There is of course some overhead but very little since, as you noted, it's smart enough to switch algorithms when faced with a file that compresses badly. It's much better than some older compressors which, worst case, could double the size of a file.

I have no idea where the 175% comes from, it makes no apparent sense either way you consider it.
This User Gave Thanks to Corona688 For This Post:
# 5  
Old 07-16-2013
From gzip documentation -
Code:
    Compression is always performed, even if the compressed file
     is  slightly larger than the original. The worst case expan-
     sion is a few bytes for the gzip file header, plus  5  bytes
     every  32K  block, or an expansion ratio of 0.015% for large
     files. Note that the  actual  number  of  used  disk  blocks
     almost  never increases.  gzip preserves the mode, ownership
     and timestamps of files when compressing or decompressing.

What you show does not indicate 175% of anything, the documentation says 'No way'


Code:
$ jim> bc -l
20915558979 / 22213027840
 .94158973417106202123

So, I think:
1. Your problem is that whatever you used to do the % calculation overflowed integers and gave garbage results. Those are 20GB files.

2. In fact, you appear to have had about 6% compression.

3. That 6% was probably because there were a lot of executable files/binary data files in the tar file. Those do not compress as well as text. "Normal" compression is on the order of 70%.
This User Gave Thanks to jim mcnamara For This Post:
# 6  
Old 07-16-2013
Quote:

So, I think:
1. Your problem is that whatever you used to do the % calculation overflowed integers and gave garbage results. Those are 20GB files.

2. In fact, you appear to have had about 6% compression.

3. That 6% was probably because there were a lot of executable files/binary data files in the tar file. Those do not compress as well as text. "Normal" compression is on the order of 70%.
Thanks Jim. I used
Code:
gzip -v

to get compression %. And again
Code:
gunzip -v

while decompression. Both showed same , 175.7%.

And yes it is ture, no matter what compression % it showed , file is compressed.

Thanks for the help - Corona and Jim Smilie.
# 7  
Old 07-16-2013
what versions of gzip/gunzip? Perhaps it could use an update.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Tar gzip compression rate

How good is the compression rate of gzip when you use tar with the gzip option? I am pretty amazed that a 1 GB file was reduced to 1019K. This is what I did. tar -cvf tar_test.tar.gz -T /list_of_files ls -hl -rw-r-----. 1 owner group 19 Jul 23 16:00 list_of_files -rw-r-----. 1 owner group... (7 Replies)
Discussion started by: cokedude
7 Replies

2. Linux

Best Compression technique ?

Hi all, I am working on a sample backup code, where i read the files per 7200 bytes and send it to server. Before sending to server, i compress each 7200 bytes using zlib compression algorithm using dictionary max length of 1.5 MB . I find zlib is slow. Can anyone recommend me a... (3 Replies)
Discussion started by: selvarajvss
3 Replies

3. Shell Programming and Scripting

Adding gzip compression to a connection using nc

Hello everyone, As the title suggests, I am attempting to test adding gzip compression to a connection to an application I am testing. Currently I have the application set up with httptunnel, which forwards the connection to the remote host. I would like to use a script to intercept the... (5 Replies)
Discussion started by: haggismn
5 Replies

4. UNIX and Linux Applications

Compression (gzip) for image analysis

Hi Everyone, I am a Ph.D student working on some image processing tasks and I have run into an interesting problem that I thought someone on here might have an idea about. This paper discusses a method to compare two images based upon the amount they can be compressed. Sorry, since this is my... (3 Replies)
Discussion started by: rudigarude
3 Replies

5. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies

6. UNIX for Dummies Questions & Answers

Un-compression types...

Hi Folks, As I am familiar wih both types compresion forms: gun-zip and .rpm. My questions is how do I uncompress gunz.zip type? As the .rpm I can double click and it will extract...Can someone shed some light on this and thank you... M (2 Replies)
Discussion started by: Mombo_Z
2 Replies

7. UNIX for Dummies Questions & Answers

.z file compression

Hello, I have a .z file which i understand to be UNIX. I'm on PC (no UNIX boxes) and have tried many different unzipping programs to extract this .z file, but only PicoZip has allowed me to view what is inside (telling me, i think, that it's not corrupt), but i have not been able to extract... (3 Replies)
Discussion started by: Gonecat
3 Replies

8. UNIX for Advanced & Expert Users

Predictor compression

Hi everybody, Does anybody know,whether linux-2.4.20-8 supports predictor1 compression in PPP protocol support,as i was able to see there were only two compression techniques that r being supported i.e bsd and deflate compression from "make menuconfig" under network device... (0 Replies)
Discussion started by: sriram.ec
0 Replies

9. UNIX for Advanced & Expert Users

Move with compression

How can I move a file and compress it at the same time? (8 Replies)
Discussion started by: truma1
8 Replies

10. UNIX for Dummies Questions & Answers

compression utilities

I've noticed bzip2 gives a little bit better compression than gzip. So...I'm curious...what's gives the best compression out of all the compression utilities? Thanks! (6 Replies)
Discussion started by: jalburger
6 Replies
Login or Register to Ask a Question