Recursive FTP -- here at last. Post: 32386

Sponsored Content

Top Forums Shell Programming and Scripting Recursive FTP -- here at last. Post 32386 by Perderabo on Saturday 23rd of November 2002 02:43:42 PM

11-23-2002

Administrator Emeritus

Recursive FTP -- here at last.

Over the past few weeks, I saw a couple of threads requesting recursive ftp:
Copying files between 2 Unix server
ftp from NT to UNIX

I decided to try to write a script to accommodate these requests. The result is HardFeed. Here are a few examples of what it can do.

HardFeed ftpserver joeblow somedir
This will connect to "ftpserver" and the user "joeblow". It will cd to "somedir". It will look at all of the files (and only the files) there. Each remote file that does not exist in the current directory will be copied to the current directory.

HardFeed -ds ftpserver joeblow somedir
This will work as the above example did. Except now we try to create local copies of any symbolic links or directories that we found in "somedir". Again, though, we will not overwrite any pre-existing object.

HardFeed -rds ftpserver joeblow somedir
Now we will create copies of any remote directories and desend into them. This will copy an entire directory tree. (except that it continues to ignore special files, pipes, etc.)

HardFeed -rs ftpserver joeblow somedir
This is similiar, except we only desend into pre-existing directories. You can use this to copy part of a directory structure. Just pre-create the few directories that you want to copy.

HardFeed -rdsm ftpserver joeblow somedir
This will copy a directory tree, but this time it will try to duplicate the mode (permissions) on each remote object that is duplicated.

HardFeed -rdsf ftpserver joeblow somdir
The -f is "freshen". Again it copies a directory tree. But if we have a local file and a remote file, the timestamps are compared. The remote file will overwrite the the local file if the remote file was newer.

There is more, including support for microsoft ftp servers that has been lightly tested. See the script itself for more info.

Last edited by Perderabo; 11-23-2002 at 04:01 PM..

These 2 Users Gave Thanks to Perderabo For This Post:

Perderabo

View Public Profile for Perderabo

Find all posts by Perderabo

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Recursive FTP

I am trying to write a recursive FTP script and have come to a point where I need to test if the file is either a normal ascii file or a directory. My question is how do I test if the file is either ascii or directory.

2. UNIX for Dummies Questions & Answers

recursive effect!!

I run the following command in some of my folders... and ended up with a huge mess!! find . -type f -exec perl -e 's/blabla/zzzxxxx/gi' -p -i.bak {} \; I had to kill the process and later when I checked with one of my folders.. ls vaditerm.dt.bak vaditerm.dt.bak.bak...

3. Shell Programming and Scripting

perl + Net::FTP::Recursive

Problem: It will not advance to the next user in the list. It always dies right after it sends the 2/2 files from the first users dir. $USERLIST="/export/home/mxdooley/perl_ftp/userlist"; $USER_DIR="/export/home/mxdooley/perl_ftp/homes";...

4. Shell Programming and Scripting

recursive rcp

I wrote a shell script (AIX) to extract the file "/rep1/toto" from all the hosts referred in a list and send them to one local directory named ~/$host-$file with the hostname as prefix rcp -p user@host:/rep1/$file ~/$host-$file where file = toto ==> it works ! I would do the same thing...

5. UNIX for Advanced & Expert Users

recursive sorting

In the ls command, -t option and -R option dont work simultaneously. ls -t ---> lists the files with sorting based on file date ls -R ---> lists the files recursively. How to make utilize both in the same command.? I want to sort the recursive files listing..

6. Cybersecurity

Recursive SFTP

Hello, I need to transfer files from Serve1 to Server2. Previously I was using scp command. Now I have to use sftp (due to audit issues). The problem with sftp is (atleast to my level of knowledge) we cannot transfer dirs (and files within that dir). Is there a way to solve this? Looks like...

7. UNIX for Dummies Questions & Answers

recursive wc on a directory?

Hi all, I need to count the number of lines in all the files under a directory (several levels deep). I am feeling extremely dumb, but I don't know how to do that. Needless to say, I am not a shell script wiz... Any advice? thanks in advance!

8. UNIX for Dummies Questions & Answers

Recursive Permissions???

Is there anyway that I can change permissions on a directory and all its sub-directories and files using one single "chmod" command??

9. UNIX for Dummies Questions & Answers

recursive search and ftp

Could someone help me in recursive search and ftp'ing the files to remote server? The host machine will have /dir1/dira/list_of_files1 /dir1/dirb/list_of_files2 /dir1/dirc/list_of_files3 . . . so., I need to search from dir1 recursively (only one level down) and find all the files that...

10. OS X (Apple)

Search recursive

before posting, I have tried to find my answer elsewhere. no luck. I need to find a file buried in a folder somewhere. Master folder has 10 sub folders. each sub folder has folders too. I found this but it does nothing I am on Mac and use Applescript. do shell script "find...

LEARN ABOUT HPUX

cmcp

cmcp(1m)																  cmcp(1m)

NAME

       cmcp - Copy files between Serviceguard nodes

SYNOPSIS

       cmcp [-rv] [src_node:]src_file [...]  [dest_node:]dest_file

DESCRIPTION

       cmcp copies files and/or directories between Serviceguard nodes.

       cmcp  is  limited in several ways. cmcp is best suited for situations where other remote copy facilities are not available. For a more full
       featured tool for copying files between Serviceguard nodes, see ccp(1m) provided by Distributed Systems Administration Utilities.

       To use this command, the local user must have the root access role on each of the involved nodes.  For  more  information  on  Serviceguard
       access control policies, see the manual, Managing Serviceguard.

       Paths  may be relative to the current working directory or specified absolutely.  The current working directory on a remote node is assumed
       to be the same as the local node's directory.  Copies between two remote hosts are permitted.

       If a file or directory already exists on the destination node, the destination node's copy will be overwritten with no prompting.

       The cmcp command is not suited for large files or large numbers of files and may result in a heavy load on  the	command  and  Serviceguard
       daemons. Other tools such as ccp should be considered for these purposes.

   Options And Arguments
	      -r   Recursively copy entire directories.

	      -v   Verbose output will be displayed.

	      src_node
		   Serviceguard node to retrieve the file from. Defaults to the local node.

	      src_file
		   Path to source file on src_node.

	      dest_node
		   Serviceguard node to copy file to. Defaults to the local node.

	      dest_file
		   Path to copy src_file to on the dest_node.

RETURN VALUE

       cmcp returns the following values:

	       0   Successful completion.
	       1   Command failed.

EXAMPLES

       Copy the local file /etc/hosts to remote node node1

	      cmcp /etc/hosts node1:/etc/hosts

       Copy the remote file /etc/passwd on node1 locally

	      cmcp node1:/etc/passwd /tmp/passwd.poe

       Copy a local script directory to a remote node

	      cmcp -r /etc/cmcluster/scripts node1:/etc/cmcluster

AUTHOR

       cmcp was developed by HP.

SEE ALSO

       rcp(1), scp(1), ccp(1m), cmsync(1m), cmexec(1m), cmdo(1m), cmquerycl(1m), cmviewcl(1m)

						      Requires Optional Serviceguard Software						  cmcp(1m)

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Recursive FTP

Discussion started by: aslamg

2. UNIX for Dummies Questions & Answers

recursive effect!!

Discussion started by: sskb

3. Shell Programming and Scripting

perl + Net::FTP::Recursive

Discussion started by: Optimus_P

4. Shell Programming and Scripting

recursive rcp

Discussion started by: Nicol