Tar gzip compression rate


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users Tar gzip compression rate
# 1  
Old 09-19-2019
Tar gzip compression rate

How good is the compression rate of gzip when you use tar with the gzip option? I am pretty amazed that a 1 GB file was reduced to 1019K. This is what I did.

Code:
tar -cvf tar_test.tar.gz -T /list_of_files
ls -hl
-rw-r-----. 1 owner group    19 Jul 23 16:00 list_of_files
-rw-r-----. 1 owner group 1019K Sep 19 02:17 tar_test.tar.gz

This User Gave Thanks to cokedude For This Post:
# 2  
Old 09-19-2019
Hi,

The compression efficiency really depends on what is being compressed, in general high compression ratios of 90% plus seem to be available on large text files. But most compression utilities tend to give only a few percent when compressing binaries or populated database tables.

I dont think I've seen the 99.9% that you've achieved here, but I have seen 90% regularly on text files and I'm sure that on one occasion a few years ago an Oracle Database that I was migrating reduced by a huge amount just can't rmember by how much.

From the example above, I'm not sure how you compressed the file, are you sure that everything is there?

Regards

Gull04

Last edited by gull04; 09-19-2019 at 05:07 AM.. Reason: Correct Typo's
This User Gave Thanks to gull04 For This Post:
# 3  
Old 09-19-2019
Looks strange to me...
You did not use the GNU tar gzip option -z at all.
So content is not compressed.

Can we see output of
Code:
file tar_test.tar.gz

Does tar -tvf tar_test.tar.gz display the files you wanted to tar on standard output ?

Are you sure you used right from file location ?
The /list_of_files is under root(/), is that your working folder ?

Compression ratio depends on what are you compressing.

Regards
Peasant.
# 4  
Old 09-19-2019
Quote:
Originally Posted by Peasant
Looks strange to me...
You did not use the GNU tar gzip option -z at all.
So content is not compressed.

Can we see output of
Code:
file tar_test.tar.gz

Does tar -tvf tar_test.tar.gz display the files you wanted to tar on standard output ?

Are you sure you used right from file location ?
The /list_of_files is under root(/), is that your working folder ?

Compression ratio depends on what are you compressing.

Regards
Peasant.
I checked the GNU version of tar and the only way to auto compress using file extension magic is to use the "a" flag:

Code:
You can also let GNU tar select the compression program based on the
suffix of the archive file name. This is done using ‘--auto-compress' (‘-a')
command line option. For example, the following invocation will use bzip2
for compression:

$ tar caf archive.tar.bz2 .


GNU tar manual in PDF attached as reference.
This User Gave Thanks to Neo For This Post:
# 5  
Old 09-19-2019
Hi Neo,

Thanks for the feedback, I was about to go and look for an update to tar as mine didn't seem to work that way.

Regards

Gull04
# 6  
Old 09-19-2019
Quote:
Originally Posted by gull04
Hi Neo,

Thanks for the feedback, I was about to go and look for an update to tar as mine didn't seem to work that way.

Regards

Gull04
Welcome.

I wanted to test further but I am in the middle of making a new "how to use code tags and format posts video" for unix.com, and have been having some issues with rendering today; so my entire day has been wasted rendering video. But the good news is that "eventually" we will have a new "how to use code tags" video Smilie

Sorry, if my post(s) on this GNU tar issue are not fully tested, as I have a lot of balls up in the air right now, video, code changes, mobile CSS for the site and more..... but I did finally take a closer look at the GNU tar PDF while video was rendering.

Cheers
This User Gave Thanks to Neo For This Post:
# 7  
Old 09-19-2019
Sparse file?
Code:
ls -las t*
  8 -rw-rw-r-- 1 user group 1024000002 Sep 19 13:50 test
972 -rw-rw-r-- 1 user group     993894 Sep 19 13:51 t.gz


And, gzip invoked by tar takes account of "The obsolescent environment variable GZIP" for e.g. the -# --fast --best compression method.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Show Percentage Compression in GZIP

Hi, I used gzip command to compress a huge tar file. But I saw that compression % was more than 100%. It might have inflated instead , probably because tar file is already packed properly. So I thought of unzippping it. Now after unzip I expected the tar file to be of less size than... (12 Replies)
Discussion started by: vinay4889
12 Replies

2. Shell Programming and Scripting

Adding gzip compression to a connection using nc

Hello everyone, As the title suggests, I am attempting to test adding gzip compression to a connection to an application I am testing. Currently I have the application set up with httptunnel, which forwards the connection to the remote host. I would like to use a script to intercept the... (5 Replies)
Discussion started by: haggismn
5 Replies

3. UNIX and Linux Applications

Compression (gzip) for image analysis

Hi Everyone, I am a Ph.D student working on some image processing tasks and I have run into an interesting problem that I thought someone on here might have an idea about. This paper discusses a method to compare two images based upon the amount they can be compressed. Sorry, since this is my... (3 Replies)
Discussion started by: rudigarude
3 Replies

4. AIX

GNU Tar with compression for AIX 4.2 already compiled ?

Is there a GNU Tar available with compression option available for AIX 4.2 that I can directly use (not having to compile anything, simple installation) ? (1 Reply)
Discussion started by: Browser_ice
1 Replies

5. AIX

AIX 4.2: tar using compression ?

Hi, is there a way to use compression on the TAR command running on a AIX 4.2 ? I did a "man tar" but did not see mentions of using compression, nor how to even find out the tar version. I want to look into ways of reducing the amount of time to do backups. One backup is dumping database... (9 Replies)
Discussion started by: Browser_ice
9 Replies

6. UNIX for Dummies Questions & Answers

tar and gzip

Hi, I would like to have a combined gzip and tar that will compress and create multiple output tar.gz files. I want to have multiple files output because i cannot create an archive because there is no more space on my harddisk. I cannot transfer it locally because of slow connection. I want to... (3 Replies)
Discussion started by: tungaw2004
3 Replies

7. UNIX for Advanced & Expert Users

tar/gzip/gz...which one to use?

P0251WLADC.svm_wl1 > /svm_wl1/billing/data/server/archive/ALLEVT $ du -k FEB2006 22050224 FEB2006 As you can see,i have a folder called "FEB2006" which is around 22 GB. i guess zip or compress wont work...( i don know how do we compress a folder) i wished to use ""tar" ( i suppose... (5 Replies)
Discussion started by: abhijeetkul
5 Replies

8. UNIX for Dummies Questions & Answers

TAR and GZIP help

Hi, There are 700 .pdf files in a certain directory on the server and I need to TAR them first and then compress them using GZIP to free up the space. The combined size of the .pdf files is 3gb. However, there is only 1gb of free space on the server. So as you can see when I try to TAR these... (3 Replies)
Discussion started by: VandeMatram
3 Replies

9. UNIX for Dummies Questions & Answers

can i tar and gzip in one liner ?

hello can i combine this 2 commands in one liner command? (1 Reply)
Discussion started by: umen
1 Replies

10. UNIX for Dummies Questions & Answers

tar compression - please help

I'm backing up one of my sites, and will be moving to another host. This was an expermintal site, and many of the directories won't be needed. What tar command is needed to exclude directories from the compressed file? Could you also give an example. also this might be the same comand, but -... (4 Replies)
Discussion started by: Taylor
4 Replies
Login or Register to Ask a Question