How to gzip files "on fly" before copying


 
Thread Tools Search this Thread
Top Forums UNIX for Advanced & Expert Users How to gzip files "on fly" before copying
# 1  
Old 07-17-2012
How to gzip files "on fly" before copying

Hello,
I want to gzip some files before copying to remote host. There is no freespace on source host so it needs to be perfomed within one-liner. I tried the following but it didn't work
Code:
gzip -c -9 all_rvds.xml |ssh targethost "dd of=/tmp/all_rvds.xml.gz"

Code:
cat all_rvds.xml |gzip -c9 |ssh targethost "dd of=/tmp/all_rvds.xml.gz"

What is the right way?
# 2  
Old 07-17-2012
The rarely-seen useful use of cat. No need to use dd.
Code:
gzip < inputfile | ssh username@host 'cat > outputfile'

You can transfer multiples by streaming tar.

Code:
tar -zcf - /path/to/files | ssh username@host 'tar -C /path/to/dest -zxf -

# 3  
Old 07-17-2012
Code:
[35]adm@sourcehost:~> gzip < all_rvds.xml | ssh targethost "cat > /tmp/all_rvds.xml.gz"
##########
WELCOME MESSAGE
##########
adm@targethost's password:
bash: line 1: syntax error near unexpected token `)'
bash: line 1: ØiPì][s²~ß_Á½pKÎÖðéD·%)G©­-M1v%&iÇþ÷9ºI.^íº*6)}h F÷tãàïßînk_GÓÙÍýøýO4 ?ÕFãáýõÍøÓû.úwñOµ¿ÿü·ùÍÇá½úïèvt7ÏkóïÑû=²Wûó~6îÔ?Ü~û<»ùHÂÉ^íf2¸¾f³÷{2h,ʽÚä~:¿I¢~ùËl4Íyü}x{÷m¶·e/'

# 4  
Old 07-17-2012
Works fine here.

Something funny is going on with your remote host. You gave it a line of script to execute, there's no reason for it to try and execute the data you gave it as a script.
# 5  
Old 07-17-2012
How to find out why it is going such way?
# 6  
Old 07-17-2012
It would help to know what your system is, and what the system you're connecting to is.

You could try ssh username@host 'sh -c "command I want to execute"' to try and force it to not use stdin for commands...
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script - Print an ascii file using specific font "Latin Modern Mono 12" "regular" "9"

Hello. System : opensuse leap 42.3 I have a bash script that build a text file. I would like the last command doing : print_cmd -o page-left=43 -o page-right=22 -o page-top=28 -o page-bottom=43 -o font=LatinModernMono12:regular:9 some_file.txt where : print_cmd ::= some printing... (1 Reply)
Discussion started by: jcdole
1 Replies

2. Shell Programming and Scripting

Delete all log files older than 10 day and whose first string of the first line is "MSH" or "<?xml"

Dear Ladies & Gents, I have a requirement to delete all the log files in /var/log/test directory that are older than 10 days and their first line begin with "MSH" or "<?xml" or "FHS". I've put together the following BASH script, but it's erroring out: for filename in $(find /var/log/test... (2 Replies)
Discussion started by: Hiroshi
2 Replies

3. Shell Programming and Scripting

awk command to replace ";" with "|" and ""|" at diferent places in line of file

Hi, I have line in input file as below: 3G_CENTRAL;INDONESIA_(M)_TELKOMSEL;SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL My expected output for line in the file must be : "1-Radon1-cMOC_deg"|"LDIndex"|"3G_CENTRAL|INDONESIA_(M)_TELKOMSEL"|LAST|"SPECIAL_WORLD_GRP_7_FA_2_TELKOMSEL" Can someone... (7 Replies)
Discussion started by: shis100
7 Replies

4. Shell Programming and Scripting

Store Host lookup in variable ("on the fly")

Hi, I'm new here. I was wondering why I can't store a host lookup in a variable. for line in $(< blacklist) do STOREIP=host $line; if ]; then $line >> blacklist2; else $line >> blacklist3; fi done Result: "ip" command not found .. so how would I store the host lookup in the... (2 Replies)
Discussion started by: sOliver
2 Replies

5. Shell Programming and Scripting

"Join" or "Merge" more than 2 files into single output based on common key (column)

Hi All, I have working (Perl) code to combine 2 input files into a single output file using the join function that works to a point, but has the following limitations: 1. I am restrained to 2 input files only. 2. Only the "matched" fields are written out to the "matched" output file and... (1 Reply)
Discussion started by: Katabatic
1 Replies

6. Shell Programming and Scripting

Delete files older than "x" if directory size is greater than "y"

I wrote a script to delete files which are older than "x" days, if the size of the directory is greater than "y" #!/bin/bash du -hs $1 while read SIZE ENTRY do if ; then find $1 -mtime +$2 -exec rm -f {} \; echo "Files older than $2 days deleted" else echo "free Space available"... (4 Replies)
Discussion started by: JamesCarter
4 Replies

7. UNIX for Advanced & Expert Users

Why is wget copying my directory tree with some files with "@"?

I'm using wget 1.11.4 on Cygwin 1.5.25. I'm trying to recursively download a directory tree, which is the root of a javadoc tree. This is approximately the command line I tried: wget -x -p -r http://<host>/.../apidoc When it finished, it seemed like it downloaded... (0 Replies)
Discussion started by: dkarr
0 Replies

8. UNIX for Dummies Questions & Answers

Copying multiple files with "If Then Else" logic

I need some suggestions on how to write the code to copy multiple files rather than duplicating the code multiple times. Example: I have four files that need to go throught this logic in the same way. Do I have to duplicate this chunk of code four times or can I built the logic in the same set... (2 Replies)
Discussion started by: madhunk
2 Replies

9. UNIX for Dummies Questions & Answers

Does "gzip" have a no prompt option on it for overwriting if file exists?

So I dont enounter things like: gzip: /sometimename.gz already exists; do you wish to overwrite (y or n)? Want to add it into a script and if there is a file aready there to just overwrite it, otherwise the script will hang unless there is manual intervention. (1 Reply)
Discussion started by: LordJezo
1 Replies

10. UNIX for Dummies Questions & Answers

Copying all files of type "pdf"

Hi, Within a shell script I'm trying to copy all the files in a directory structure to another folder but only the files of type "pdf" I have made a start but I can't work out how to finish it. find $ATEBUILD/doc/user -name "*.pdf" | xargs cp THEN WHAT? I hope this makes sense! Rob (6 Replies)
Discussion started by: milage
6 Replies
Login or Register to Ask a Question