Best Compression technique ?


 
Thread Tools Search this Thread
Operating Systems Linux Best Compression technique ?
# 1  
Old 03-25-2013
Best Compression technique ?

Hi all,

I am working on a sample backup code, where i read the files per 7200 bytes and send it to server. Before sending to server, i compress each 7200 bytes using zlib compression algorithm using dictionary max length of 1.5 MB . I find zlib is slow.

Can anyone recommend me a compression technique to satisfy my above requirements.
I saw lz4 compression, but it does not support dictionary in it.

or Is there any way to improve the existing zlib algorithm ..??

./Selva..,
# 2  
Old 03-26-2013
Are you doing this as a shell script or a C/C++ application?
# 3  
Old 03-26-2013
RE:

Hi fpmurphy,

Its for my C++ application.

Thanks for your reply..!!!


./Selva.,
# 4  
Old 03-26-2013
Hi.

There are some benchmarks at Maximum Compression (lossless data compression software)

You'll probably need to poke around a bit amongst the different tables, and possibly look for a balance between speed and space.

Let us know what you choose -- best wishes ... cheers, drl
Login or Register to Ask a Question

Previous Thread | Next Thread

8 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Password Obscuring Technique

Hi, We have a unix shell script which tries login to database. The user name and password to connect to database is stored in a file connection.sql. Now connection.sql has contents def ora_user =&1 CONNECT A_PROXY/abc123@DEV01 When on UNIX server we connect to database and set spool on... (7 Replies)
Discussion started by: Gangadhar Reddy
7 Replies

2. Shell Programming and Scripting

file compression

I'am looking for script (or tool) that would compress all files with given extension in all subdirectory. Important part is that every one file have to end in separate archive whit it's own name. Eaven if I could point multiple file in one directory and compress them it would be ok. I' am... (1 Reply)
Discussion started by: Demerzel
1 Replies

3. Shell Programming and Scripting

Folder compression

Hi Guys, totally new to scripting. I have a huge list of foleders. I want to tar the contents of each folder inside the same folder and the name of the tar file is the same name as the folder. how would I go about this in scirpt? cheers Viper (3 Replies)
Discussion started by: brownviper1966
3 Replies

4. AIX

Tape compression

Hello everyone I want to use compression in my tape when I backup some file. For example I have several files that use 50gb. If I backup this I need to use two cartridge because without compression I can backup 36gb. My question is with flag I need to use to compress and I can use 72gb in... (2 Replies)
Discussion started by: lo-lp-kl
2 Replies

5. Shell Programming and Scripting

Best search technique

I have a snippet file with the shown below: data file 1 2 1 3 1 3 4 2 3 2 2 1 2 2 5 1 3 2 3 2 2 3 1 4 Actual file has approx 50 Millions such lines with bigger number (9 Replies)
Discussion started by: chakrapani
9 Replies

6. Shell Programming and Scripting

How to avoid CR after compression

Hi all, I am having few files which needs to be concted into a single file and then it is compressed and FTPed from the UNIX server to the Windows server. For the above purpose i am using gzip command to compress the files after concetenation. And i am FTP ing the compressed file in the... (3 Replies)
Discussion started by: Codesearcher
3 Replies

7. UNIX for Dummies Questions & Answers

FORK/EXEC technique

Hi! Can someone explain me exactly this technique? Why a process (PARENT) creates a copy of itself with FORK (CHILD)? What's the reason of this behaviour? Sorry, but I cannot understand the logic behind it. Thanks. (4 Replies)
Discussion started by: marshmallow
4 Replies

8. UNIX for Dummies Questions & Answers

Difference Technique's???

Is there any better way of doing this? I only want to find a status of a diff, ie diff the file and return to me whether it is different or not or non-existant. This example works, however I think it could be less messier: workd=`pwd`;find $workd -name "*.sum" | while read line ; do... (1 Reply)
Discussion started by: Shakey21
1 Replies
Login or Register to Ask a Question