scp/untar .tar file in parallel issue


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting scp/untar .tar file in parallel issue
# 1  
Old 10-21-2011
scp/untar .tar file in parallel issue

Hi Guys,

I am facing a strange issue
while doing parallel (using & for background) scp/untar operation from my unix box to multiple unix boxes...
I am getting
tar : unexpected EOF in archive error
the code is as follows.,,,
Code:
for i in 10 
do
sh -c "scp <command> ; ssh tar -xf  <tar> -C <location>" &
done

Please help me get rid of this,
Also please comment on reliability of SCP as i read that the issue can be due to SCP?

---------- Post updated at 01:53 PM ---------- Previous update was at 01:52 PM ----------

The code is just a skeleton, i am sure the syntax is correct
# 2  
Old 10-21-2011
What is your OS version of the server (where you scp from) and the clients you scp and untar.
Perhaps there is a compatibility issue.
Does it work if you do it manualy ?

I belive issue is not due to SCP, it's reliable.
You do have ssh keys exchanged between the server and clients ?

Use set -x to debug the shell code.
# 3  
Old 10-21-2011
scp/untar .tar file in parallel issue

Its the same version of OS and it doesnt happen always but occassionally it happens, and manually when i do it it wrks fine... mostly it wrks but sometimes it gives this? is it due to the matter that i m running it in parallel ?
# 4  
Old 10-21-2011
Can you confirm that the tar archive isn't being created while it's being copied ?

A complete description would be lovely, as much detail as you can spare about the complete action and desired result.
# 5  
Old 10-21-2011
It kind of depends what <command> is.

What are you trying to do? Extract the same archive on lots of machines at the same time you're copying it? You should stream that, not copy it, or it may hit eof before it's done.

Code:
for SERVER in server1 server2 server3
do
        ssh username@$SERVER tar -C /path/to/dest -xf - < /path/to/file.tar &
done

wait

Running 9 simultaneous ssh/scp/whatever sessions may not be a good idea, you might want to limit that.

If you can install it, you can use udpcast to stream one thing to 9 servers.

Code:
for x in 1 2 3
do
        ssh username@server$X udp-receiver '|' tar -C /path/to/dest -xf - 2> /dev/null &
done

udp-sender < file.tar
wait

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Solaris

How to copy a tar file on a series of remote hosts and untar it on those hosts?

Am trying to copy a tar file onto a series of remote hosts and untar it at the destination. Need to do this without having to do multiple ssh. Actions to perform within a single ssh session via shell script - copy a file - untar at destination (remote host) OS : Linux RHEL6 (3 Replies)
Discussion started by: sankasu
3 Replies

2. UNIX for Dummies Questions & Answers

How to Untar tar.gz to particular directory

Hi Guys, I am able to untar a tar.gz file. But it is again extracting the tar file to further child directory. I even tried the below command to untar it to particular directory. tar -zxvf gme_dly_sls_20120515035335.txt.tar.gz -C /sites/VSTAR/site/live/ftp/GMEUROPE I am getting the below... (4 Replies)
Discussion started by: mac4rfree
4 Replies

3. UNIX for Advanced & Expert Users

scp/untar in parallel issue tar : Unexpected EOF in archive

Hi Guys, I am facing a strange issue while doing parallel (using & for background) scp/untar operation from my unix box to multiple unix boxes... I am getting tar : unexpected EOF in archive error the code is as follows.,,, for i in 10 do sh -c "scp <command> ; ssh tar -xf <tar> -C... (1 Reply)
Discussion started by: mihirvora16
1 Replies

4. Red Hat

tar/untar over ssh

I use red hat linux. I have two linux server . I want to use tar over ssh to tar and untar the file. The server A , have IP 10.1.1.a ,there is dir a and contain files. The server B have IP 10.1.1.b , there is dir b and contain file . So, in above case ,how can I tar over ssh sunc that the file... (0 Replies)
Discussion started by: chuikingman
0 Replies

5. Shell Programming and Scripting

untar .tar.gz file to a specific file

Hi, I'll get a tarred file from a remote location in the format of .tar.gz which my program needs to un-tar it into a specific destination foler and a file Incoming file/local/server/source/path/remote_file.txt.tar.gz Extracted file/local/server/destination/path/un-tar/stage_file.txt ... (5 Replies)
Discussion started by: dips_ag
5 Replies

6. UNIX for Dummies Questions & Answers

untar a tar file

how can I untar a file without extracting it? sample: file.tar to file thanks, lara (1 Reply)
Discussion started by: lhareigh890
1 Replies

7. UNIX for Dummies Questions & Answers

tar and untar commands

I have a script to ftp, archive and delete files. I used tar command to archive files from a list and then all files were removed from name1/name2/name2/. find /name1/name2/name2/ -name "*.txt" -print > filelist.log while read line do if ; then tar cvf $tarfile $line else ... (3 Replies)
Discussion started by: Lenora2009
3 Replies

8. Shell Programming and Scripting

Script to untar latest tar file

I am trying to put together a script that will check for the latest file in a directory then extract it. The extraction and the scheduling I can do, but am not sure how to get it to check for the latest file. These files are uploaded every evening by an external party and the previous days files... (3 Replies)
Discussion started by: stheologo
3 Replies

9. UNIX for Advanced & Expert Users

Tar utility (untar a .tar file) on VxWorks

Hi All Can someone pls guide me if there any utility to compress file on windows & uncompress on vxworks I tried as - - compressed some folders on windows ... i created .tar ( to maintain directory structure ) and compressed to .gz format. - on VxWorks i have uncompressed it to .tar... (1 Reply)
Discussion started by: uday_01
1 Replies

10. UNIX for Dummies Questions & Answers

Untar a TAR file at different location

Hi, I want to UNTAR a TAR file at different location. Is it possible? My TAR file contains the files with absolute path. Malay (5 Replies)
Discussion started by: malaymaru
5 Replies
Login or Register to Ask a Question

Featured Tech Videos