Hi again
I know that in most cases using ssh or ssl would be fine. Indeed I have both stunnel (ssl) and ssh versions of this connection working fine, with ssl being slightly faster.

However the overall objective is to demonstrate suricata's ability to decompress gzip data on the fly and detect patterns vs regular methods like iptables/layer7, which just look at the plaintext pattern rather than putting it together.

To do this, I will of course need a connection which uses gzip compression.

If I can get this nc/gzip connection working, I then show that a regular layer7 rule will not be able to detect regex pattens that the test program ouputs. However it is expected that suricata will be able to detect the patterns.

In the case of using http to form the connection, you are right, it is obvious from the traffic pattern that it isn't a standard http connection. However this isn't relevant as yet.

I am attempting firstly to get the nc port forwarding working on both the server and host. Once this is working I will look again at putting in gzip commands. I have a similar command on both machines, however on the server, things do not work properly. I have the server set up with the http tunnel server to receive, then forward packets to port 8008. I am then using this command (after mkfifo /tmp/rev)

Code:

while true ; do cat /tmp/rev | nc -l 8008 | nc localhost 80 > /tmp/rev ; done

with the test server program listening on port 80.

I notice when I enter this command, cpu usage increases dramatically and I get a lot of errors; something isn't right

Code:

TCP: time wait bucket table overflow

If I use this command on the host (with different ports), it forwards correctly between the test program and the tunnel application.

Edit: Well that part is fixed. Turns out the server was using a different version of nc. I had to add "-p" to the listening nc. Now I just need to add gzip/gzip -d in the right places.

---------- Post updated at 08:30 PM ---------- Previous update was at 05:47 PM ----------

Well I have made a little progress. It seems as though there is a problem, possibly to do with gzip not knowing when to stop compressing, although I could be wrong. I am using the following command on both sides. The link works fine if I remove the gzip parts.

Code:

while true ; do cat /tmp/rev | gzip -v | nc -l (-p )8080 | gzip -v | nc localhost 80 | gzip -d > /tmp/rev ; done

Can anyone assist here? If its necessary to do so, is there a way to inform gzip to compress until x bytes, then output? If what I am attempting to achieve is not possible, please let me know so I don't waste too much time!

Again, any help would be greatly appreciated.Thanks