Making output of data from remote hosts smaller


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Making output of data from remote hosts smaller
# 1  
Old 10-19-2015
Making output of data from remote hosts smaller

so i'm doing something similar to this:

Code:
ssh myname@remotehost 'tail -800 /var/log/some.log'

Now, as you can see, this is a lot of data to be passing back and forth over a network.

Is there anything i can do to make the output smaller (zip it on the fly, compress?) and then when the data gets back to my local box, i can unpackage it to its original?

im looking for a one liner type of thing.
# 2  
Old 10-19-2015
Simple enough:

Code:
ssh -T myname@remotehost 'tail -800 /var/log/some.log | gzip' | gunzip

The -T prevents it from allocating a pseudo-terminal and doing any escape sequences, making the stream binary-safe.
This User Gave Thanks to Corona688 For This Post:
# 3  
Old 10-19-2015
You can also do ssh -C but that's not always supported.
This User Gave Thanks to Corona688 For This Post:
# 4  
Old 10-19-2015
ssh supports compression if you tell it to, using the -C flag
From man ssh:
Code:
     -C      Requests compression of all data (including stdin, stdout, stderr, and data for
             forwarded X11 and TCP connections).  The compression algorithm is the same used
             by gzip(1), and the “level” can be controlled by the CompressionLevel option
             for protocol version 1.  Compression is desirable on modem lines and other slow
             connections, but will only slow down things on fast networks.  The default
             value can be set on a host-by-host basis in the configuration files; see the
             Compression option.

# 5  
Old 10-19-2015
Some ssh implementations support compression, OpenSSH for instance. Those that don't may just ignore it or throw errors. The remote host also has to support it.
This User Gave Thanks to Corona688 For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

awk remote multiple hosts print remote hostname and output

Hi all, i'm trying to gether multiple pattern on remote hosts, and trying to print hostname and the pattern, ssh remoteserver1 -C 'hostname 2>&1;cat /var/log/server1.log | awk -F ";" '"'"'{ print " "$2" "$5}'"'"'| sort | uniq -c | sort -g -r ' The output is the following, remoteserver1 ... (8 Replies)
Discussion started by: charli1
8 Replies

2. Shell Programming and Scripting

Grep remote multiple hosts output to local server

Hello all, i'm trying to create a report by greping a pattern on multiple remote hosts and creta a simple report, actually i did this, is ther any better way to do this. #!/bin/bash for host in `cat RemoteHosts` do ssh $host -C 'hostname 2>&1; grep ERROR /var/log/WebServer.log.2019-09-21... (0 Replies)
Discussion started by: charli1
0 Replies

3. Solaris

How to copy a tar file on a series of remote hosts and untar it on those hosts?

Am trying to copy a tar file onto a series of remote hosts and untar it at the destination. Need to do this without having to do multiple ssh. Actions to perform within a single ssh session via shell script - copy a file - untar at destination (remote host) OS : Linux RHEL6 (3 Replies)
Discussion started by: sankasu
3 Replies

4. Shell Programming and Scripting

SSH to remote hosts in shell scripting

Hi There, I have a file contaning some 100 servers names one by one the file called redhat_servers.txt I want to prepare a script where it should give me the host name and kernal version. I wrote like this, #!/bin/bash while read line do ssh $line "uname -nr" done <... (3 Replies)
Discussion started by: kumar85shiv
3 Replies

5. UNIX for Advanced & Expert Users

Port Time outs on Remote Hosts

I monitor all my servers using Nagios. Now, for Nagios to run certain checks, it has to connect to remote hosts via a certain port. Lets say the port is 8090. Now, when Nagios connects to a remote host (for an example) via port 8090, and it has to run 6 checks (scripts) on that remote host, i... (2 Replies)
Discussion started by: SkySmart
2 Replies

6. UNIX for Advanced & Expert Users

remote hosts and wide output?

Hello forum: I am curious about some output that I get using an alias <command> on a remote host and I wondered if someone could point me in the right direction. Symptoms: Using "ssh -qi /path/to/key root@som.ipa.ddr.ess mail" (or variation of via alias) only gives a partial textual... (7 Replies)
Discussion started by: Habitual
7 Replies

7. Debian

Launch remote gui apps in remote hosts

Hi, I've been looking for a way to execute a console program (is in windows but by now I accept the linux way) from a linux machine, but this program has to be opened in the remote side. Linux machine acts only as a "signaling" host. My program has to open the camera in the remote side, but only... (7 Replies)
Discussion started by: zauberberg
7 Replies

8. UNIX for Dummies Questions & Answers

How to Know is a Remote hosts have open port

Hi gurus of unix!!!!, I have a little question. I nedd your helps The scenarios is the following I have tree equipment that are installed in different places. I use a carrier to interconnect the equipment. Some Port's (TCP) need to be open for an application that must be function correctly. For... (3 Replies)
Discussion started by: andresguillen
3 Replies

9. UNIX for Advanced & Expert Users

ftp into remote hosts

Hi all, Have used ftp to transfer files from remote host to localhost. I was wondering how can I ftp into remote hosts. for example from a unix box, connect to an external server and then ftp that file into mainframe ? I would like to avoid pulling it to unix box and then ftping to mainframe.... (1 Reply)
Discussion started by: kodermanna
1 Replies

10. UNIX for Dummies Questions & Answers

Copy files between remote hosts

Don't know if its correct to write into this topic but how can I copy files from a Windows machine to a UNIX system? (2 Replies)
Discussion started by: agasamapetilon
2 Replies
Login or Register to Ask a Question