Sponsored Content
Top Forums Shell Programming and Scripting Recursive FTP -- here at last. Post 32386 by Perderabo on Saturday 23rd of November 2002 02:43:42 PM
Old 11-23-2002
Recursive FTP -- here at last.

Over the past few weeks, I saw a couple of threads requesting recursive ftp:
Copying files between 2 Unix server
ftp from NT to UNIX

I decided to try to write a script to accommodate these requests. The result is HardFeed. Here are a few examples of what it can do.

HardFeed ftpserver joeblow somedir
This will connect to "ftpserver" and the user "joeblow". It will cd to "somedir". It will look at all of the files (and only the files) there. Each remote file that does not exist in the current directory will be copied to the current directory.

HardFeed -ds ftpserver joeblow somedir
This will work as the above example did. Except now we try to create local copies of any symbolic links or directories that we found in "somedir". Again, though, we will not overwrite any pre-existing object.

HardFeed -rds ftpserver joeblow somedir
Now we will create copies of any remote directories and desend into them. This will copy an entire directory tree. (except that it continues to ignore special files, pipes, etc.)

HardFeed -rs ftpserver joeblow somedir
This is similiar, except we only desend into pre-existing directories. You can use this to copy part of a directory structure. Just pre-create the few directories that you want to copy.

HardFeed -rdsm ftpserver joeblow somedir
This will copy a directory tree, but this time it will try to duplicate the mode (permissions) on each remote object that is duplicated.

HardFeed -rdsf ftpserver joeblow somdir
The -f is "freshen". Again it copies a directory tree. But if we have a local file and a remote file, the timestamps are compared. The remote file will overwrite the the local file if the remote file was newer.

There is more, including support for microsoft ftp servers that has been lightly tested. See the script itself for more info.

Last edited by Perderabo; 11-23-2002 at 04:01 PM..
These 2 Users Gave Thanks to Perderabo For This Post:
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Recursive FTP

I am trying to write a recursive FTP script and have come to a point where I need to test if the file is either a normal ascii file or a directory. My question is how do I test if the file is either ascii or directory. (1 Reply)
Discussion started by: aslamg
1 Replies

2. UNIX for Dummies Questions & Answers

recursive effect!!

I run the following command in some of my folders... and ended up with a huge mess!! find . -type f -exec perl -e 's/blabla/zzzxxxx/gi' -p -i.bak {} \; I had to kill the process and later when I checked with one of my folders.. ls vaditerm.dt.bak vaditerm.dt.bak.bak... (2 Replies)
Discussion started by: sskb
2 Replies

3. Shell Programming and Scripting

perl + Net::FTP::Recursive

Problem: It will not advance to the next user in the list. It always dies right after it sends the 2/2 files from the first users dir. $USERLIST="/export/home/mxdooley/perl_ftp/userlist"; $USER_DIR="/export/home/mxdooley/perl_ftp/homes";... (2 Replies)
Discussion started by: Optimus_P
2 Replies

4. Shell Programming and Scripting

recursive rcp

I wrote a shell script (AIX) to extract the file "/rep1/toto" from all the hosts referred in a list and send them to one local directory named ~/$host-$file with the hostname as prefix rcp -p user@host:/rep1/$file ~/$host-$file where file = toto ==> it works ! I would do the same thing... (6 Replies)
Discussion started by: Nicol
6 Replies

5. UNIX for Advanced & Expert Users

recursive sorting

In the ls command, -t option and -R option dont work simultaneously. ls -t ---> lists the files with sorting based on file date ls -R ---> lists the files recursively. How to make utilize both in the same command.? I want to sort the recursive files listing.. (1 Reply)
Discussion started by: fermisoft
1 Replies

6. Cybersecurity

Recursive SFTP

Hello, I need to transfer files from Serve1 to Server2. Previously I was using scp command. Now I have to use sftp (due to audit issues). The problem with sftp is (atleast to my level of knowledge) we cannot transfer dirs (and files within that dir). Is there a way to solve this? Looks like... (1 Reply)
Discussion started by: MohanTJ
1 Replies

7. UNIX for Dummies Questions & Answers

recursive wc on a directory?

Hi all, I need to count the number of lines in all the files under a directory (several levels deep). I am feeling extremely dumb, but I don't know how to do that. Needless to say, I am not a shell script wiz... Any advice? thanks in advance! (13 Replies)
Discussion started by: bimba17
13 Replies

8. UNIX for Dummies Questions & Answers

Recursive Permissions???

Is there anyway that I can change permissions on a directory and all its sub-directories and files using one single "chmod" command?? (5 Replies)
Discussion started by: the_red_dove
5 Replies

9. UNIX for Dummies Questions & Answers

recursive search and ftp

Could someone help me in recursive search and ftp'ing the files to remote server? The host machine will have /dir1/dira/list_of_files1 /dir1/dirb/list_of_files2 /dir1/dirc/list_of_files3 . . . so., I need to search from dir1 recursively (only one level down) and find all the files that... (1 Reply)
Discussion started by: brahmi
1 Replies

10. OS X (Apple)

Search recursive

before posting, I have tried to find my answer elsewhere. no luck. I need to find a file buried in a folder somewhere. Master folder has 10 sub folders. each sub folder has folders too. I found this but it does nothing I am on Mac and use Applescript. do shell script "find... (2 Replies)
Discussion started by: sbrady
2 Replies
ncftpls(1)						      General Commands Manual							ncftpls(1)

NAME
ncftpls - Internet file transfer program for scripts SYNOPSIS
ncftpls [options] ftp://url.style/host/path/name/ OPTIONS
Command line flags: -m Use a machine readable list format, if the server supports it. This requires that the server software support the MLSD extensions, and many implementations do not have these features. -1 Most basic format, one item per line. -l Long list format. -C Columnized list format. This is the default list format. -R Recurse all subdirectories while listing. -a Show all files, if server allows it (as in "/bin/ls -a"). -i XX Filter the listing (if server supports it) with the wildcard XX. -x -XX Set the ls flags to use on the server. -u XX Use username XX instead of anonymous. -p XX Use password XX with the username. -P XX Use port number XX instead of the default FTP service port (21). -d XX Use the file XX for debug logging. -t XX Timeout after XX seconds. -E Use regular (PORT) data connections. -F Use passive (PASV) data connections. The default is to use passive, but to fallback to regular if the passive connection fails or times out. -r XX Redial a maximum of XX times until connected to the remote FTP server. -W XX Send raw FTP command XX after logging in. -X XX Send raw FTP command XX after each file transferred. -Y XX Send raw FTP command XX before logging out. The -W, -X, and -Y options are useful for advanced users who need to tweak behavior on some servers. For example, users accessing mainframes might need to send some special SITE commands to set blocksize and record format information. For these options, you can use them multiple times each if you need to send multiple commands. For the -X option, you can use the cookie %s to expand into the name of the file that was transferred. -o XX Set advanced option XX. This option is used primarily for debugging. It sets the value of an internal variable to an integer value. An example usage would be: -o useFEAT=0,useCLNT=1 which in this case, disables use of the FEAT command and enables the CLNT command. The available variables include: usePASV, useSIZE, useMDTM, useREST, useNLST_a, useNLST_d, useFEAT, useMLSD, useMLST, useCLNT, useHELP_SITE, useSITE_UTIME, STATfileParamWorks, NLSTfileParamWorks, require20, allowProxyForPORT, doNotGetStartCWD. DESCRIPTION
The purpose of ncftpls is to do remote directory listings using the File Transfer Protocol without entering an interactive shell. This lets you write shell scripts or other unattended processes that can do FTP. The default behavior is to print the directory listing in columnized format (i.e. ls -CF), but that is not very useful for scripting. This example uses the -1 flag, to print one file per line: $ ncftpls -1 ftp://ftp.ncftp.com/pub/ncftp/ You can also do a remote "ls -l", by using "ncftpls -l". If you want to try other flags, you have to use them with the -x flag. For exam- ple, if you wanted to do a remote "ls -lrt", you could do this: $ ncftpls -x "-lrt" ftp://ftp.ncftp.com/pub/ncftp/ By default the program tries to open the remote host and login anonymously, but you can specify a username and password information like you can with ncftpget or ncftpput. Note that the standard specifies that URL pathnames are are relative pathnames. For FTP, this means that URLs specify relative pathnames from the start directory, which for user logins, are typically the user's home directory. If you want to use absolute pathnames, you need to include a literal slash, using the "%2F" code for a "/" character. Examples: $ ncftpls -u linus ftp://ftp.kernel.org/%2Fusr/src/ $ ncftpls ftp://steve@ftp.apple.com/%2Fetc/ DIAGNOSTICS
ncftpls returns the following exit values: 0 Success. 1 Could not connect to remote host. 2 Could not connect to remote host - timed out. 3 Transfer failed. 4 Transfer failed - timed out. 5 Directory change failed. 6 Directory change failed - timed out. 7 Malformed URL. 8 Usage error. 9 Error in login configuration file. 10 Library initialization failed. 11 Session initialization failed. AUTHOR
Mike Gleason, NcFTP Software (http://www.ncftp.com). SEE ALSO
ncftpput(1), ncftpget(1), ncftp(1), ftp(1), rcp(1), tftp(1). LibNcFTP (http://www.ncftp.com/libncftp/). ncftpls NcFTP Software ncftpls(1)
All times are GMT -4. The time now is 06:21 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy