04-14-2008
Quote:
Originally Posted by
vijaykrc
Thank you for the help...
I may be moving around 5000 files and i cannot tar them... i need to send individually...
I am benefiting around 3 hrs if i send then at a time using the * and if i send then individually its taking that 3-4 hrs more for the 5000 files.
The way I would approach this if the contents of the source directory were likely to change while the copy process is running is to create a shell function to copy one individual file and then to delete it if the copy was successful. I would then start the process by creating a list of all the current files, passing them one by one to the function. This will ensure that only files that were copied correctly get deleted. I know that this will take extra time and resources as each individual file will require a new connection to be set up, but it is, in my opinion the safest way. Of course you will need to set up key authorisation rather than password auth as that would be tedious in the extreme for 5000 files.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
I need to transfer multiple files using SCP between two solaris machines. Can somebody explain how to achieve that ? (3 Replies)
Discussion started by: parthum
3 Replies
2. Shell Programming and Scripting
Hi Frdz
I have a problem like.
I need to transfer a file from source to destination (different systems with different IPs) using "scp" command and before transfer the file i have to check the file is available in destination or not, if it is there no need to transfer, otherwise we have to... (5 Replies)
Discussion started by: KiranKumarKarre
5 Replies
3. UNIX for Dummies Questions & Answers
I have a shell script which uses SCP command to transfer the files from one server to another server. The files are getting transferred successfully, but the problem is the files transferred to the destination server didnot have the permissions as that of the files on the source server.
Command... (5 Replies)
Discussion started by: kumarm
5 Replies
4. UNIX for Advanced & Expert Users
I am not able to throw a file from server173 to server067
i.e.
wlsuser@server173> scp /tmp/harsha.txt wlsuser@server067:/tmp
fails
However, I am able to pull a file from server173 onto server067's /tmp dir
wlsuser@server067> scp wlsuser@server173:/tmp/harsha.txt /tmp... (2 Replies)
Discussion started by: shifahim
2 Replies
5. UNIX for Advanced & Expert Users
Whenever I transfer file through SCP between two server it gives below given message.....Is there a way to avoid it....
Target server : newyork
$ scp ABC27801.iue newyork:./iABC/x0017801.iue
Message
=====
This system is for the use of authorized users only. Individuals using this... (1 Reply)
Discussion started by: Pash
1 Replies
6. UNIX for Advanced & Expert Users
I have 3 AIX server namely - Server 1 , Server 2 and Server 3.
And have done SCP setup between Server 1 and Server 2 so that i dont have to give password when i transfer file from Server 1 to Server 2 by setting public key between the server.
Q1. If the unix password of the target server... (3 Replies)
Discussion started by: Pash
3 Replies
7. UNIX for Advanced & Expert Users
On unix AIX server, when I am trying to transfer file from one directory to another directory on the same server through a program(where i call the script) it gives error "Lost Connection". (5 Replies)
Discussion started by: Pash
5 Replies
8. AIX
I am testing the following command to transfer a file from my server (AIX 5.2) to another server. I was able to generate the keys and sent them the public key.
scp -v -P 4030 /home/lawson/.ssh/jimtest.txt someuser@some.ftpsite.net:/Inbound/jimtest.txt > jimtest_out.txt 2>&1
Based on... (3 Replies)
Discussion started by: jyoung
3 Replies
9. Shell Programming and Scripting
I am trying to transfer a zip file of around 30 MB in my automation script using scp from system A to B.
When I manually do scp, file is complete transferred but when automation shell script runs it, zip file is not completely transferred.
Stack Trace while doing manual :
Executing:... (1 Reply)
Discussion started by: Shaishav Shah
1 Replies
10. UNIX for Beginners Questions & Answers
Hi All,
Need some help regarding file transfer between server.
Suppose we have system-A and system-B. To transfer file from system-A to system-B we usually share the
public keys of system-A to system-B and do scp/sftp to transfer a file.
Is it possible that public key of system-B can be... (3 Replies)
Discussion started by: abhi_123
3 Replies
SCP(1) BSD General Commands Manual SCP(1)
NAME
scp -- secure copy (remote file copy program)
SYNOPSIS
scp [-346BCpqrv] [-c cipher] [-F ssh_config] [-i identity_file] [-l limit] [-o ssh_option] [-P port] [-S program] [[user@]host1:]file1 ...
[[user@]host2:]file2
DESCRIPTION
scp copies files between hosts on a network. It uses ssh(1) for data transfer, and uses the same authentication and provides the same secu-
rity as ssh(1). scp will ask for passwords or passphrases if they are needed for authentication.
File names may contain a user and host specification to indicate that the file is to be copied to/from that host. Local file names can be
made explicit using absolute or relative pathnames to avoid scp treating file names containing ':' as host specifiers. Copies between two
remote hosts are also permitted.
The options are as follows:
-3 Copies between two remote hosts are transferred through the local host. Without this option the data is copied directly between the
two remote hosts. Note that this option disables the progress meter.
-4 Forces scp to use IPv4 addresses only.
-6 Forces scp to use IPv6 addresses only.
-B Selects batch mode (prevents asking for passwords or passphrases).
-C Compression enable. Passes the -C flag to ssh(1) to enable compression.
-c cipher
Selects the cipher to use for encrypting the data transfer. This option is directly passed to ssh(1).
-F ssh_config
Specifies an alternative per-user configuration file for ssh. This option is directly passed to ssh(1).
-i identity_file
Selects the file from which the identity (private key) for public key authentication is read. This option is directly passed to
ssh(1).
-l limit
Limits the used bandwidth, specified in Kbit/s.
-o ssh_option
Can be used to pass options to ssh in the format used in ssh_config(5). This is useful for specifying options for which there is no
separate scp command-line flag. For full details of the options listed below, and their possible values, see ssh_config(5).
AddressFamily
BatchMode
BindAddress
CanonicalDomains
CanonicalizeFallbackLocal
CanonicalizeHostname
CanonicalizeMaxDots
CanonicalizePermittedCNAMEs
CertificateFile
ChallengeResponseAuthentication
CheckHostIP
Ciphers
Compression
ConnectionAttempts
ConnectTimeout
ControlMaster
ControlPath
ControlPersist
GlobalKnownHostsFile
GSSAPIAuthentication
GSSAPIDelegateCredentials
HashKnownHosts
Host
HostbasedAuthentication
HostbasedKeyTypes
HostKeyAlgorithms
HostKeyAlias
HostName
IdentitiesOnly
IdentityAgent
IdentityFile
IPQoS
KbdInteractiveAuthentication
KbdInteractiveDevices
KexAlgorithms
LogLevel
MACs
NoHostAuthenticationForLocalhost
NumberOfPasswordPrompts
PasswordAuthentication
PKCS11Provider
Port
PreferredAuthentications
ProxyCommand
ProxyJump
PubkeyAcceptedKeyTypes
PubkeyAuthentication
RekeyLimit
SendEnv
ServerAliveInterval
ServerAliveCountMax
StrictHostKeyChecking
TCPKeepAlive
UpdateHostKeys
UsePrivilegedPort
User
UserKnownHostsFile
VerifyHostKeyDNS
-P port
Specifies the port to connect to on the remote host. Note that this option is written with a capital 'P', because -p is already
reserved for preserving the times and modes of the file.
-p Preserves modification times, access times, and modes from the original file.
-q Quiet mode: disables the progress meter as well as warning and diagnostic messages from ssh(1).
-r Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal.
-S program
Name of program to use for the encrypted connection. The program must understand ssh(1) options.
-v Verbose mode. Causes scp and ssh(1) to print debugging messages about their progress. This is helpful in debugging connection,
authentication, and configuration problems.
EXIT STATUS
The scp utility exits 0 on success, and >0 if an error occurs.
SEE ALSO
sftp(1), ssh(1), ssh-add(1), ssh-agent(1), ssh-keygen(1), ssh_config(5), sshd(8)
HISTORY
scp is based on the rcp program in BSD source code from the Regents of the University of California.
AUTHORS
Timo Rinne <tri@iki.fi>
Tatu Ylonen <ylo@cs.hut.fi>
BSD May 3, 2017 BSD