Sponsored Content
Top Forums Shell Programming and Scripting Adding gzip compression to a connection using nc Post 302603989 by haggismn on Friday 2nd of March 2012 01:19:27 PM
Old 03-02-2012
Adding gzip compression to a connection using nc

Hello everyone,
As the title suggests, I am attempting to test adding gzip compression to a connection to an application I am testing. Currently I have the application set up with httptunnel, which forwards the connection to the remote host.

I would like to use a script to intercept the connection between the application and httptunnel, gzip the data that is there, and then forward it on. On the other side, the opposite will happen with gzip -d, with the resulting data passed on to the server application.

However I need some help, as I am fairly new to some of the things which are required to do this, like mkfifo, as well as piping outputs. I think there may be some other problems implementing such an idea with multiple connections, but that is a problem for later.

I have created a connection which works fine for sending gz files, and a while loop to keep things going.

Code:
while true ; do nc -l 3333 | gzip -d ; done

and on the other side, simply

Code:
 nc 192.168.20.84 3333 < a.gz

Can someone assist in developing this so that I can use this method to compress the connection on one side ,and decompress on the other side?

The following is an attempt to do what what needs to be done on one side, but on other, I am not so sure about decompressing and passing data on. httptunnel is listening on 8080 of course.

Code:
#!/bin/sh
mkfifo /tmp/gz.tmp
while true; do
cat gz.tmp | gzip | nc -l -k localhost 9123 | nc localhost 8080 ;
done

Secondly, I know that this will only compress data travelling in one direction. I would like to have the same idea operating in reverse, from the server back to the host. Would this be possible? This might be quite complicated, due to dynamic source ports. Having gzip know what to do with the data might be another problem. Perhaps attempting "gzip -d" first, then if it gives a fail code, just run "gzip". Is there a cleaner way of detecting gzip compression, if it is needed?

Any help would be greatly appreciated.

Last edited by haggismn; 03-02-2012 at 02:34 PM..
 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Connection reset by peer..closing connection

Hello I'm facing the above problem while doing a performance run. I've a script which I'm launching from my windows desktop using mozilla. The script will invoke backend action on a Solaris host which in turn feeds the records to a driver located on a linux box(Cent OS). What's happening is... (1 Reply)
Discussion started by: subramanyab
1 Replies

2. Solaris

Solaris 10 ftp connection problem (connection refused, connection timed out)

Hi everyone, I am hoping anyone of you could help me in this weird problem we have in 1 of our Solaris 10 servers. Lately, we have been having some ftp problems in this server. Though it can ping any server within the network, it seems that it can only ftp to a select few. For most servers, the... (4 Replies)
Discussion started by: labdakos
4 Replies

3. Shell Programming and Scripting

file compression

I'am looking for script (or tool) that would compress all files with given extension in all subdirectory. Important part is that every one file have to end in separate archive whit it's own name. Eaven if I could point multiple file in one directory and compress them it would be ok. I' am... (1 Reply)
Discussion started by: Demerzel
1 Replies

4. UNIX for Advanced & Expert Users

gzip vs pipe gzip: produce different file size

Hi All, I have a random test file: test.txt, size: 146 $ ll test.txt $ 146 test.txt Take 1: $ cat test.txt | gzip > test.txt.gz $ ll test.txt.gz $ 124 test.txt.gz Take 2: $ gzip test.txt $ ll test.txt.gz $ 133 test.txt.gz As you can see, gzipping a file and piping into gzip... (1 Reply)
Discussion started by: hanfresco
1 Replies

5. UNIX and Linux Applications

Compression (gzip) for image analysis

Hi Everyone, I am a Ph.D student working on some image processing tasks and I have run into an interesting problem that I thought someone on here might have an idea about. This paper discusses a method to compare two images based upon the amount they can be compressed. Sorry, since this is my... (3 Replies)
Discussion started by: rudigarude
3 Replies

6. UNIX for Advanced & Expert Users

How keep running a program n an another computer via a connection ssh when the connection is closed?

Hi everybody, I am running a program on a supercomputer via my personal computer through a ssh connection. My program take more than a day to run, so when I left work with my PC I stop the connection with the supercomputer and the program stop. I am wondering if someone know how I can manage... (2 Replies)
Discussion started by: TomTomGre
2 Replies

7. Linux

Best Compression technique ?

Hi all, I am working on a sample backup code, where i read the files per 7200 bytes and send it to server. Before sending to server, i compress each 7200 bytes using zlib compression algorithm using dictionary max length of 1.5 MB . I find zlib is slow. Can anyone recommend me a... (3 Replies)
Discussion started by: selvarajvss
3 Replies

8. Shell Programming and Scripting

Show Percentage Compression in GZIP

Hi, I used gzip command to compress a huge tar file. But I saw that compression % was more than 100%. It might have inflated instead , probably because tar file is already packed properly. So I thought of unzippping it. Now after unzip I expected the tar file to be of less size than... (12 Replies)
Discussion started by: vinay4889
12 Replies

9. UNIX for Advanced & Expert Users

Compression with openssl

Hi , 1-I need to know please if it's possible to compress using openssl? Here is the version used: openssl version -a OpenSSL 0.9.7d 17 Mar 2004 (+ security fixes for: CVE-2005-2969 CVE-2006-2937 CVE-2006-2940 CVE2006-3738 CVE-2006-4339 CVE-2006-4343 CVE-2007-5135 CVE-2008-5077... (3 Replies)
Discussion started by: Eman_in_forum
3 Replies

10. UNIX for Advanced & Expert Users

Tar gzip compression rate

How good is the compression rate of gzip when you use tar with the gzip option? I am pretty amazed that a 1 GB file was reduced to 1019K. This is what I did. tar -cvf tar_test.tar.gz -T /list_of_files ls -hl -rw-r-----. 1 owner group 19 Jul 23 16:00 list_of_files -rw-r-----. 1 owner group... (7 Replies)
Discussion started by: cokedude
7 Replies
ns_zlib(3aolserver)					     AOLserver Zlib Extension					       ns_zlib(3aolserver)

__________________________________________________________________________________________________________________________________________________

NAME
ns_zlib - Zlib compression support SYNOPSIS
ns_zlib compress string ns_zlib gunzip file ns_zlib gzip string ns_zlib gzipfile file ns_zlib uncompress string _________________________________________________________________ DESCRIPTION
The ns_zlib command enables compressing and uncompressing of strings or files. The command is available if the nszlib.so module is loaded into AOLserver or the libnszlib.so, nszlib.dll, or libnszlib.dylib dynamic library is loaded using the load command in a suitable tclsh such as nstclsh. ns_zlib compress string This command compresses the given string and returns a Tcl byte array object with the compressed data. ns_zlib gunzip file This command uncompresses the contents of the given gzipped file and returns a string as the result. ns_zlib gzipfile file This command is similar to the gzip shell routines, compressing the given file into a new file with the .gz extension. If success- ful, the original uncompressed file is deleted. ns_zlib uncompress bytearray This command takes a byte array object which includes compressed data and returns an uncompressed string object. EXAMPLES
The following examples demonstrate compressing and uncompressing a string; # Compress Tcl string set test "This is test string" set data [ns_zlib compress $test] set test [ns_zlib uncompress $data] --> returns "This is test string" # Compress the string into gzip format set gzip [ns_zlib gzip $test] # Save as gzip file set fd [open /tmp/test.gz w] fconfigure $fd -translation binary -encoding binary puts -nonewline $fd $gzip close $fd # Uncompress gzipped file set test [ns_zlib gunzip /tmp/test.gz] --> returns "This is test string" SEE ALSO
ns_adp_ctl(n), load(n) KEYWORDS
GZIP, compress, uncompress AOLserver 4.5 ns_zlib(3aolserver)
All times are GMT -4. The time now is 11:08 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy