Sponsored Content
Full Discussion: Confirming Remote Copy
Top Forums Shell Programming and Scripting Confirming Remote Copy Post 302470408 by dlam on Wednesday 10th of November 2010 04:17:29 AM
Old 11-10-2010
Confirming Remote Copy

Hey guys.

I'm knocking up a script, a part of which will be copying some pretty large files to some remote servers. With the size of the files I need to make sure that they don't lose any data / corrupt on the way (it's unlikely, but these are business critical).

Obviously I could use the exit code to confirm the copy completed successfully but I assume that it wouldn't know if any data was lost.

The option I've thought of is to do a checksum on the file being copied, and do a remote checksum once the copy is completed and compare the two. But was just wondering if anyone could think of a neater way of doing it.

Thanks.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Remote file copy

I facing a problem with Unix command "rcp". I unable to perform a rcp between host machines. I have religiously followed the man pages, but still unable to solve the problem. Do i check for anything to perform this command? Pls help....thanks =) (3 Replies)
Discussion started by: lchunl
3 Replies

2. HP-UX

help me to copy remote file

I want to copy dump generated from oracle database to my local DAT drive. Currently I am copying remote file to local drive thru rcp command and later copy it to local DAT. Pls. help me in this. Thanks Man Mohan email address removed (8 Replies)
Discussion started by: manmohan73
8 Replies

3. Shell Programming and Scripting

Confirming Syntax - IF statement.

Hi All, Has been a while since I was last on, so I hope everyone has been doing fine. ;) Would like to know if the below IF statement syntax is correct for a ksh environment. It's been pushed into live as someone had deleted the development copy(!); not withstanding that, the statement now... (3 Replies)
Discussion started by: Cameron
3 Replies

4. UNIX for Dummies Questions & Answers

Copy files between remote hosts

Don't know if its correct to write into this topic but how can I copy files from a Windows machine to a UNIX system? (2 Replies)
Discussion started by: agasamapetilon
2 Replies

5. AIX

Confirming command

Guy's I'm trying to create script to execute some critical commands in AIX like the below ... stopsrc -s qdaemon stopsrc -s sshd stopsrc -s lpd stopsrc -s nfsd I want between each command to ask me like this " are you sure to stop qdaemon " " are you sure to stop ssh" " are... (3 Replies)
Discussion started by: Mr.AIX
3 Replies

6. Shell Programming and Scripting

Remote Copy Script

Hi All, My first post on the Forums! I've just whipped up a quick script to slightly automate a log file retrieval process for work and to be completely honest my BASH scirpting knownledge is extremely limited as is my understanding of a lot of linux commands. Basically the scripts purpose... (3 Replies)
Discussion started by: effektd
3 Replies

7. UNIX for Advanced & Expert Users

Using remote rsync, but copy locally?

I'm looking to use rsync to compare remote files and to copy the diff to a local directory, rather than transfer over the net. The net connection is not fast enough to transfer these files (~1.8TB) and I'd like to sneakernet them instead. Possible? (4 Replies)
Discussion started by: dfbills
4 Replies

8. Shell Programming and Scripting

Confirming a string

Hi Guys, I have files that are sent to me in this format: Copy of USer Database updated 23 JANUARY 2013.csv I am developing a script to load these files to my database automatically whenever I get these files: How do I verify these files in an if statement if I do have these files? ... (5 Replies)
Discussion started by: Phuti
5 Replies

9. UNIX for Dummies Questions & Answers

Remote copy through shellscript

I am using RedHat 4.4 Version Operating System. I want to copy one file from source to destination.I don't want to give a password when I am executing the below shell script. code: #!/bin/bash scp -r file.zip 192.168.1.5:/root/ Here it asking a password to enter,Is there any way give a... (2 Replies)
Discussion started by: kannansoft1985
2 Replies

10. Shell Programming and Scripting

Copy from local to remote

Hi I need a advice for writing simple bash script, I have a file pod.txt which contains source location and remote location: /mnt/infile/20141103/701_0001.png/remote/tmp/pk21730/p0330223723074.png /mnt/infile/20141103/203_0001.png/remote/tmp/pk21731/p0330223723081.png and I must copy ... (6 Replies)
Discussion started by: primo102
6 Replies
SVN::Notify::Mirror::SSH(3pm)				User Contributed Perl Documentation			     SVN::Notify::Mirror::SSH(3pm)

NAME
SVN::Notify::Mirror::SSH - Mirror a repository path via SSH SYNOPSIS
Use svnnotify in post-commit: svnnotify --repos-path "$1" --revision "$2" --handler Mirror::SSH --to "/path/to/www/htdocs" [--svn-binary /full/path/to/svn] [[--ssh-host remote_host] [--ssh-user remote_user] [--ssh-tunnel 10.0.0.2] [--ssh-identity /home/user/.ssh/id_rsa]] or better yet, use SVN::Notify::Config for a more sophisticated setup: #!/usr/bin/perl -MSVN::Notify::Config=$0 --- #YAML:1.0 '': PATH: "/usr/bin:/usr/local/bin" 'path/in/repository': handler: Mirror to: "/path/to/www/htdocs" 'some/other/path/in/repository': handler: Mirror::SSH to: "/path/to/remote/www/htdocs" ssh-host: "remote_host" ssh-user: "remote_user" ssh-tunnel: "10.0.0.2" ssh-identity: "/home/user/.ssh/id_rsa" DESCRIPTION
Keep a directory in sync with a portion of a Subversion repository. Typically used to keep a development web server in sync with the changes made to the repository. This directory can either be on the same box as the repository itself, or it can be remote (via SSH connection). USAGE
Depending on whether the target is a "Local Mirror" or a Remote Mirror, there are different options available. All options are available either as a commandline option to svnnotify or as a hash key in SVN::Notify::Config (see their respective documentation for more details). Working Copy on Mirror Because 'svn export' is not able to be consistently updated, the sync'd directory must be a full working copy, and if you are running Apache, you should add lines like the following to your Apache configuration file: # Disallow browsing of Subversion working copy # administrative directories. <DirectoryMatch "^/.*/.svn/"> Order deny,allow Deny from all </DirectoryMatch> The files in the working copy must be writeable (preferrably owned) by the user identity executing the hook script (this is the user identity that is running Apache or svnserve respectively). Local Mirror Please see " SVN::Notify::Mirror " for details. Remote Mirror Used for directories not located on the same machine as the repository itself. Typically, this might be a production web server located in a DMZ, so special consideration must be paid to security concerns. In particular, the remote mirror server may not be able to directly access the repository box. NOTE: be sure and consult "Remote Mirror Pre-requisites" before configuring your post-commit hook. o ssh-host This value is required and must be the hostname or IP address of the remote host (where the mirror directories reside). o ssh-user This value is optional and specifies the remote username that owns the working copy mirror. o ssh-identity This value may be optional and should be the full path to the local identity file being used to authenticate with the remote host. If you are setting the ssh-user to be something other than the local user name, you will typically also have to set the ssh-identity. o ssh-tunnel If the remote server does not have direct access to the repository server, it is possible to use the tunneling capabilities of SSH to provide temporary access to the repository. This works even if repository is located internally, and the remote server is located outside of a firewall or on a DMZ. The value passed for ssh-tunnel should be the IP address to which the local repository service is bound (when using svnserve). This will tunnel port 3690 from the repository box to localhost:3690 on the remote box. This must also be the way that the original working copy was checked out (see below). To tunnel some other port, for example when using Apache/mod_dav, ssh-tunnel should be the entire mapping expression, as described in the OpenSSH documentation under the "-R" option (remote port forwarding). For most sites, passing "8080:10.0.0.2:80" will work (which will tunnel port 80 from the repository to port 8080 on the remote client). If you are using SSL with Apache, you can use e.g. "80443:10.0.0.2:443". For example, see "Remote Mirror Pre-requisites" and after step #6, perform the following additional steps (when using svnserve): # su - localuser $ ssh -i .ssh/id_rsa remote_user@remote_host -R3690:10.0.0.2:3690 $ cd /path/to/mirror/working/copy $ svn co svn://127.0.0.1/repos/path/to/files . where 10.0.0.2 is the IP address hosting the repository service. For the same configuration when using Apache/mod_dav, do this instead: # su - localuser $ ssh -i .ssh/id_rsa remote_user@remote_host -R8080:10.0.0.2:80 $ cd /path/to/mirror/working/copy $ svn co http://127.0.0.1:8080/repos/path/to/files . o ssh-options If you have any other options that you would like to pass to the ssh client (for example to change the default SSH port), you can pass extra options using this parameter. Be sure that you pass it a string that has ssh long option/value pairs separated by a space, or short options without any space at all. Internally, parameter is split on spaces and passed in the @Net::SSH::options array. Remote Mirror Pre-requisites Before you can configure a remote mirror, you need to produce an SSH identity file to use: 1. Log in as repository user Give the user identity being used to execute the hook scripts (the user running Apache or svnserve) a shell and log in as that user, e.g. "su - svn"; 2. Create SSH identity files on repository machine Run "ssh-keygen" and create an identity file (without a password). 3. Log in as remote user Perform the same steps as #1, but this time on the remote machine. This username doesn't have to be the same as in step #1, but it must be a user with full write access to the mirror working copy. 4. Create SSH identity files on remote machine It is usually more efficient to go ahead and use "ssh-keygen" to create the .ssh folder in the home directory of the remote user. 5. Copy the public key from local to remote Copy the .ssh/id_dsa.pub (or id_rsa.pub if you created an RSA key) to the remote server and add it to the .ssh/authorized_keys for the remote user. See the SSH documentation for instructions on how to configure 6. Confirm configuration As the repository user, confirm that you can sucessfully connect to the remote account, e.g.: # su - local_user $ ssh -i .ssh/id_rsa remote_user@remote_host This is actually a good time to either check out the working copy or to confirm that the remote account has rights to update the working copy mirror. If the remote server does not have direct network access to the repository server, you can use the tunnel facility of SSH (see ssh-tunnel above) to provide access (e.g. through a firewall). Once you have set up the various accounts, you are ready to set your options. AUTHOR
John Peacock <jpeacock@cpan.org> COPYRIGHT
Copyright (c) 2005-2008 John Peacock This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
SVN::Notify, SVN::Notify::Config, SVN::Notify::Mirror POD ERRORS
Hey! The above document had some coding errors, which are explained below: Around line 278: You forgot a '=back' before '=head1' perl v5.14.2 2012-07-04 SVN::Notify::Mirror::SSH(3pm)
All times are GMT -4. The time now is 01:31 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy