Transfer large number of files host to host


 
Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Transfer large number of files host to host
# 8  
Old 10-21-2010
What Operating System and version is the source computer?
What Operating System and version is the object computer?
# 9  
Old 10-21-2010
One line? Use semicolons !

Which man tar has B?

If you want simple, with compression, something like this (might be scp, might be -C): scp2 -rpd +C /top_dir1 /top_dri2 /top_dir3 remote_host:/

A Google tells me rcp never has compression, but rsync does.

The single tar ssh stream:
  1. has no intermediate files, one requirement,
  2. ssh might work for root when rsh does not, somewhat an implicit hidden potential requirement,
  3. pads with nulls, a minor inefficiency cpio vs. tar (silly old command intended for tape drives),
  4. has no compression, so network speed will choke it considerably (ssh has +C or -C for gzp -? compression. but separate compression processes gives you choice and control and another process to pipeline larallelize the job),
  5. is single process, so i/o time and file search and open time will waste critical network time, not to mention single tcp socket, so there may be ack delays if max tss is not high enough, and delays if any packet is lost,
  6. ssh adds encrypt/decrypt overhead/delay,
  7. pipes and most of UNIX i/o is single buffered, so there will be time lost to blocking.

It turns out cp/*cp/rcp/scp/scp2 -r use cpio, which also has a local pass option, somewhat redundant to cp -r, but choice is good.

Last edited by DGPickett; 10-21-2010 at 10:53 AM..
This User Gave Thanks to DGPickett For This Post:
# 10  
Old 10-21-2010
Quote:
Originally Posted by Red
I have two servers, one has an empty /
What exactly does "an empty /" mean?
# 11  
Old 10-21-2010
Good question -- is this more a "clone host bypassing package installation" question?
# 12  
Old 10-21-2010
citaylor,

Thanks very, very much. I'm going to try your script between two hosts today. One (final I hope) question. Where I said *transfer* I should have said *copy*. Will the tar script copy the files or will it actually move the files and directories from the old host to the new host? Thanks again.............. Blaine



---------- Post updated at 07:59 AM ---------- Previous update was at 07:55 AM ----------




citaylor,

Thanks very, very much for the help. I'll try the tar script today. One final (I hope) question... Will the tar script *copy* the files and directories actually *move* them from the old host to the new one?
Thanks again! Blaine
# 13  
Old 10-21-2010
tar will copy - the originating host is untouched.
This User Gave Thanks to citaylor For This Post:
# 14  
Old 10-21-2010
Methyl,

Both old host and new host are RHEL 3.0.

Thanks for your assistance!
Blaine



---------- Post updated at 08:17 AM ---------- Previous update was at 08:09 AM ----------




methyl,

I guess I should have been more explicit... the old host has a subdirectory /usr/local/coremobility/... with multiple files and subdirectories and files. I need to copy theses files and subdirectories onto the new host.

Thanks again!

Blaine



---------- Post updated at 08:19 AM ---------- Previous update was at 08:17 AM ----------




DGPickett,

Please see the post I made to methyl for the description of files and subdirectories I need to transfer from old host to new host. Sorry for the ambiguity!

Thanks...

Blaine



---------- Post updated at 08:21 AM ---------- Previous update was at 08:19 AM ----------




citaylor...

Thanks.. I'll give it a shot today!

Blaine



---------- Post updated at 08:27 AM ---------- Previous update was at 08:21 AM ----------




All,

I'd love to use scp but I understand that it doesn't follow links? Also, I am root on both machines. Bandwidth and CPU on either machine and the network are not an issue. If anyone thinks rsync is the way to go, I'm game. Then the question becomes how do I use rsync?

Sorry for the seemingly idiotic questions but I'm a little lost here.

. Thanks!

Blaine



---------- Post updated at 09:19 AM ---------- Previous update was at 08:27 AM ----------




citaylor,

So, in order to copy files and directories from oldhost to newhost I should execute:

Code:
root@newhost tar -cBpf - /usr/local/coremobility | ssh root@oldhost "tar -xBpf - "

?

Or do I have the hosts backwords?

Thanks.

My apologies in advance for being so dim and if I accidentally embedded the code line into the body of the test.

Blaine
 
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Connect to target host from Source host.

Hi All, Need to connect to target host and execute a command and connect back to source host to continue with next set of command execution. Is there a utility/command using which we can connect to target host ? Please suggest. Note: Netezza database is instaled on Linux server. ... (4 Replies)
Discussion started by: Nagaraja Akkiva
4 Replies

2. Solaris

Need to recover/move diskgroup from failed host to another host

Hi All I am having VxVm on two Solaris hosts. host1 is using disk group dgHR. right now this server went down due to hardware fault. Not I need to import this dgHR into host2 server. Please let me know the procedure for the same. (1 Reply)
Discussion started by: amity
1 Replies

3. Shell Programming and Scripting

How to pass a variable from one host to another host

Hi , I have to pass a string through a variable from one host to another. Is that any any way to do it . Please help me (4 Replies)
Discussion started by: arukuku
4 Replies

4. IP Networking

ping can not recognize host but host command can

Hi, I have a weird problem. when ever I do ping command like for example ping unix.comI get the following message: # ping unix.com ping: unknown host unix.com but when I use host the computer is able to know the host. # host unix.com unix.com has address 81.17.242.186 unix.com mail is... (2 Replies)
Discussion started by: programAngel
2 Replies

5. UNIX for Advanced & Expert Users

Help! How to find the local host after few ssh hops to remote host???

I do a ssh to remote host(A1) from local host(L1). I then ssh to another remote(A2) from A1. When I do a who -m from A2, I see the "connected from" as "A1". => who -m userid pts/2 2010-03-27 08:47 (A1) I want to identify who is the local host who initiated the connection to... (3 Replies)
Discussion started by: gomes1333
3 Replies

6. Shell Programming and Scripting

running commands to remote host from centralized host

Gurus/Experts We have a centralized UNIX/Solaris server from where we can actually ssh to all other UNIX/Solaris servers...I need to write a script that reside on this centerlized server and do FileSystem monitoring (basically run df -h or -k) of other remote servers and then send an email to me... (6 Replies)
Discussion started by: anjum.suri
6 Replies

7. Solaris

How to delete the files from local host to remote host

Hi all, i am copying .gz files from production server to development server using "scp" command.my requirement is after copying .gz files i want to delete old .gz files(two days back) in development server from production server. like this way i need to delelte .log ,.z and .dmp files... (3 Replies)
Discussion started by: krishna176
3 Replies

8. Solaris

Tar files, transfer to remote host and delelte source

Hi, I´m having a problem here. I have a directory that holds many subdirs and files (by many I mean MANY thousands). What I want to do is make blocks of these files and transfer them block by block to a remote host; but once a given file is already placed in the remote host, the script must... (0 Replies)
Discussion started by: Dago
0 Replies

9. UNIX for Advanced & Expert Users

host alias not working: host not found

Hello, I am working on HP-UX , and in the /etc/hosts file we have setup an alias: aa.bb.cc.dd devmach2.unix.org devmach2 devma2v The alias devma2v does not work. Error when pinging devma2v ping: unknown host devma2v For devmach2 the ping works fine , returning the correct IP... (4 Replies)
Discussion started by: FunnyCats
4 Replies

10. IP Networking

QNX host cannot ping SCO host, vice versa

The problem I am facing now is that the QNX host could not ping the SCO host and vice versa. They are in the same domain, ie, 172.20.3.xx. As I am very new to Unix, I guess I must have missed out some important steps. Pls help... Thanx alot (2 Replies)
Discussion started by: gavon
2 Replies
Login or Register to Ask a Question