Today (Saturday) We will make some minor tuning adjustments to MySQL.

You may experience 2 up to 10 seconds "glitch time" when we restart MySQL. We expect to make these adjustments around 1AM Eastern Daylight Saving Time (EDT) US.


Tarring files to remote server


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Tarring files to remote server

Hi,

I need to tar some files in a directory to a remote server.
I need to exclude some file from this directory and then tar it over.
This is the command suggested by one article (tarring in the same server) :

Code:
tar -zcvf /tmp/mybackup.tar.gz -X exclude.txt /home/me

However it does not work as, it throws error that -z is not a valid option.

Is this the correct way to do it to change it to a .gz format :

Code:
tar -cvf -X exclude.txt - /bscsbin/lisa/product/UMOBILE/prod/bscs/scripts | gzip > dir.tar.gz

The files that I don't need is listed in "exclude.txt".

How do i do the same to send it over to a remote server?

Also, this command was suggested in an article to include files that I need in a tar format :

Code:
% find project -type f -print | \
egrep '(\.[ch]|[Mm]akefile)$' > Include

However, when I try this command (to customise according to my need) in my server, there is not output :

Code:
# find . -type f -print | egrep '(\.[jar]|\.[bash])$'

I want to list files with *.jar and *.bash extensions here.
# 2  
Quote:
Originally Posted by anaigini45
. . .
Is this the correct way to do it to change it to a .gz format :

Code:
tar -cvf -X exclude.txt - /bscsbin/lisa/product/UMOBILE/prod/bscs/scripts | gzip > dir.tar.gz

The files that I don't need is listed in "exclude.txt".
Piping into a compressing / zipping command is the / a correct approach if the z is not available. Be aware that the f option needs an archive file - drop it or specify one. For piping it needs to write to stdout.

Quote:
How do i do the same to send it over to a remote server?
The f option allows to specify an archive on a remote node - see man tar. But - you can't use that if you want to compress via a piped gzip or similar. Use e.g. scp, then.

Quote:
. . .
However, when I try this command (to customise according to my need) in my server, there is not output :

Code:
# find . -type f -print | egrep '(\.[jar]|\.[bash])$'

I want to list files with *.jar and *.bash extensions here.
Your egrep regex matches files with an .j, .a, .r, .b, .s, or .h ending. Read about "bracket expression"s in your man regex:
Quote:
A bracket expression is a list of characters enclosed in "[]".
Try
Code:
egrep '\.jar$|\.bash$'

instead.
# 3  
Hi,

When I'm doing this I tend just to do the whole thing in a single line - something like.

#> tar cf - | ssh me@server ( cd /destination ; tar xf - )

Obviously you would adjust the create command to exclude the files that you want to exclude as per your example. If the files are very large you may like to pipe through gzip or whatever compression utility, but with the speed of modern networks I seldom see the need now.

Regards

Gull04
This User Gave Thanks to gull04 For This Post:
# 4  
These were the commands I used to accomplish what I wanted :

Code:
#i=`find . \( -name '*.sh' -o -name '*.jar' -o -name '*.gz' \) -exec ls -l {} \+`
# tar cf - "$i" | ssh emoaigin@10.61.1.58 ( cd /root; /usr/contrib/bin/gzip > my.tar.gz )

I get this error :

Quote:
sh: Syntax error: `(' is not expected.
I then change the "(" in the command to "", like this :

Code:
#i=`find . \( -name '*.sh' -o -name '*.jar' -o -name '*.gz' \) -exec ls -l {} \+`
# tar cf - "$i" | ssh emoaigin@10.61.1.58 "cd /root; /usr/contrib/bin/gzip > my.tar.gz"

This time I get prompted for password, but I still get some error :

Code:
# i=`find . \( -name '*.sh' -o -name '*.jar' -o -name '*.gz' \) -exec ls -l {} \+`
# tar cf - "$i" | ssh emoaigin@10.61.1.58 "cd /root; /usr/contrib/bin/gzip > my.tar.gz"
tar: cannot open -rw-------   1 emoaigin   users            0 Mar  5 18:16 ./abc.sh
-rw-------   1 emoaigin   users            0 Mar  5 18:16 ./create.jar
-rw-------   1 emoaigin   users            0 Mar  5 18:16 ./make.gz
Password:
sh: my.tar.gz: Cannot create the specified file.
#

How do I resolve the error?
# 5  
Two generic (non- HP-UX- specific) comments:
- why the -exec ls -l action for find? You want file names, not a "long listing". Try dropping entirely.
- Does emoaigin have write access to /root - would be somewhat unusual.

Last edited by RudiC; 03-05-2018 at 08:24 AM..
# 6  
Quote:
Originally Posted by anaigini45
I need to tar some files in a directory to a remote server.
Is this the correct way to do it to change it to a .gz format :

Code:
tar -cvf -X exclude.txt - /bscsbin/lisa/product/UMOBILE/prod/bscs/scripts | gzip > dir.tar.gz

No. Actually it is lke this (i will do this step by step so that you can understand what is done):

Code:
| gzip > dir.tar.gz

This means that gzip is getting data from stdin, packing them and sending the result to stdout, which is pointed to the file dir.tar.gz.

So far, so good, but now you need to to make tar putting its output to (its) stdout, because this is what a pipe does: connecting one process' stdout to the next process' stdin. Therefore you need to tell tar to not create a tar-file, but to direct the output to stdout:

Code:
tar <other subcommands> f - |

Notice, btw., that tar is not driven by options but subcommands. Therefore the introducing "-" is wrong. Everything else is still as in the e working example you found, with the exception of the non-standard "z" command (that works only for tar, which have a "built-in" gzip):

Code:
tar cvf - -X exclude.txt /home/me | gzip > /output/file

Quote:
Originally Posted by anaigini45
How do i do the same to send it over to a remote server?
The same way you alread tried: instead of writing the file to disk (> /output/file), you can use another pipeline to move it to another server and eventually unpack it there:

Code:
tar cvf - -X exclude.txt /home/me | gzip | ssh user@host "cd /some/place ; cat - | gzip -cd | tar xf - "

Notice that this instance is one of the select few moments where a seemingly UUOC is justified. At i am not aware of any other way of making gzip look at its stdin for incoming data.

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
# 7  
@Bakunin, I tried the method you suggested, and this was the error :

Code:
# i=`find /home/emoaigin \( -name '*.sh' -o -name '*.jar' -o -name '*.gz' \)`
# tar -cvf - "$i" | gzip | ssh emoaigin@10.61.1.58 "cd /home/emoaigin ; cat - | gzip -cd | tar xf - "
sh: gzip:  not found.
tar: cannot open /home/emoaigin/abc.sh
/home/emoaigin/create.jar
/home/emoaigin/make.gz
Password:
sh: gzip:  not found.
Tar: blocksize = 0; broken pipe?
#

The error gzip not found can be resolved by putting the full path to gzip.
However, the error "tar: cannot open /home/emoaigin/abc.sh" why does it happen?
And what is this error : "Tar: blocksize = 0; broken pipe?" ?
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Script connect to remote server, not find files and exit only from remote server, but not from scrip

I have a script, which connecting to remote server and first checks, if the files are there by timestamp. If not I want the script exit without error. Below is a code TARFILE=${NAME}.tar TARGZFILE=${NAME}.tar.gz ssh ${DESTSERVNAME} 'cd /export/home/iciprod/download/let/monthly;... (3 Replies)
Discussion started by: digioleg54
3 Replies

2. Shell Programming and Scripting

Do I require remote login access to a windows server to transfer files from a UNIX server

Hi All I need to transfer a file from a UNIX server to a windows server. I saw that it is possible to do this using scp command by looking at the forum listed below: ... (2 Replies)
Discussion started by: vx04
2 Replies

3. Solaris

Script to get files from remote server to local server through sftp without prompting for password

Hi, I am trying to automate the process of fetching files from remote server to local server through sftp. I have the username and password for the remote solaris server. But I need to give password manually everytime i run the script. Can anyone help me in automating the script such that it... (3 Replies)
Discussion started by: ssk250
3 Replies

4. UNIX for Dummies Questions & Answers

Tarring files up to four days old

Hi I need help in tarring files up to four days old. I have been doing this: find . -mtime -4|xargs tar -cvf mar4.tar However, it seems like it's tarring everything but leaving those that are 4 days old and newer (which I want tarred and everything else not) ---------- Post updated at... (1 Reply)
Discussion started by: MIA651
1 Replies

5. Shell Programming and Scripting

Bash Tarring not un Tarring correctly

HI All, Im encountering behaviour that is not correct for my requirements when I untar a file. Im using the below command to tar up files from various folders to the ARCHIVE folder as below... tar -cvf "$ARCHIVE_PATH"/"$dte_tar_filename" "$LOG_PATH" "$PROCESSED_PATH2" "$ERROR_PATH" ... (5 Replies)
Discussion started by: satnamx
5 Replies

6. Shell Programming and Scripting

tarring/compressing files in Unix directory

hi guys, i'm totally new with Unix sripting and no idea how to do the scripting at all. My problem is that my boss asked me to do this: 1.) create a script that will tar or gzip the files in particular directory eg: i'm on my home directory and I need to tar/gzip the file in.. assuming... (1 Reply)
Discussion started by: montski
1 Replies

7. UNIX for Dummies Questions & Answers

tarring large no. of files

dears, I have a folder containing huge no. of files, some of them are created on AUG 16, AUG 17 and AUG 18, for example. All I want to do is tarring all the files created on a certain date, say AUG 18, in one tar file, only in one command line. So, how to feed all the files created on a certain... (4 Replies)
Discussion started by: marwan
4 Replies

8. Shell Programming and Scripting

FTP multiple files from remote server to local server

Hi, I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies

9. UNIX for Dummies Questions & Answers

tarring and gzipping dump files

Say I want to transfer several dump files from a Solaris machine onto a Win2k machine for storage. It was suggested that I tar and gzip the dump files before doing so. Is it completely necessary to use both of these utilities, or is it sufficient to compress multiple dump files into one gzip... (4 Replies)
Discussion started by: PSC
4 Replies

10. UNIX for Dummies Questions & Answers

backup : files being modified while tarring

I would like to back up several directories weekly using a cronjob. I'm not experienced in UNIX, but I would start like this: tar -cvf backup.tar dir1 dir2 dir3 Now if a file is being modified in the process it will result in an error. How can I prevent this from happening and how can I... (5 Replies)
Discussion started by: jamesbond
5 Replies

Featured Tech Videos