Recursive FTP -- here at last.


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Recursive FTP -- here at last.
# 36  
Old 11-18-2006
Quote:
Originally Posted by Perderabo
rsync depends on the berkeley remote shell command. That protocol is very insecure, perhaps the most insecure in existence. It always was unix specific and not that many unix shops still use it due to security. And I never seen a firewall configured to pass that protocol.
rsync is also available for windows aswell as novel and it can also use ssh instead of rsh.
But sometimes due to some firewall rules you can only use FTP, as I found with my last task I was given at work, so I wrote my own autoftp script. I will check yours out and might use that instead Smilie
Tornado
# 37  
Old 12-30-2006
works great

Excellent work im now migrating a big site like 600mb or so, i see that it has trouble with directories that contain subdirectories it gives an error Smilie but the rest it`s great you might want to check on that thanks a lot Smilie
# 38  
Old 12-30-2006
I have tested it extensively including with subdirectories over ten levels deep. The script works fine for me. I have used it many times to move large directory structures around.
# 39  
Old 03-23-2007
ntrans question

hello perderabo,

i'm tying to use your script with -x "ntrans \%3A \:" to convert the hex form of the ":" back to "normal". your script seems to do a check on filenames and aborts...since i'm very unexperienced with scripting: is there a way to use hardfeed with the option ntrans ??
i have been looking for a tool like hardfeed for quite a while and would really like to use...
could you please help me ?

thank you very much
ingo
# 40  
Old 03-23-2007
First of all, ntrans doesn't work like that. "ntrans \%3A \:" will convert % to : and delete any occurence of 3 and A. That might collapse into what you want if the only 3 and A characters happen to always follow a : but it seems like a dangerous way to do things. From the man page:

Quote:
ntrans [inchars [outchars]]
...
If the character's position in inchars is longer than the length of outchars, the character is deleted from the file name.
And no, hardfeed is not going to work with ntrans. hardfeed needs to send a string like "get $file" to the ftp coprocess. Then it waits until the file arrives by testing if $file exists. Once the file arrives, it may optionally do a "chmod $mode $file" to transfer the mode. Another option is retrieve the file only if the remote file is newer than the local file. All of this stuff is based on file names.

You could use hardfeed to retrieve the files, then run another script to rename them. There are many example of renaming scripts on this site and you can find them with our search function. But remember that if you rename the files, you won't be able to run hardfeed again to just retrieve changed files. hardfeed will think that any renamed files are missing.
# 41  
Old 03-24-2007
thanks for the explanation !

using hardfeed....and a renaming-script after transfer now.

another question:
i'm also using wget to transfer from unix to a xp computer...(thats where the %3A comes from Smilie ) do you know a way (tried wput....) to handle transfers like that, solely from the windows side ? without havin to rename / manipulate on the unix side ?

greetings
ingo
# 42  
Old 03-24-2007
I have used hardfeed from XP. I just downloaded the version posted in this article and tried it again. It works for me. hardfeed does not require a true ksh, pdksh is good enough. I have installed Microsoft Services for Unix and I run hardfeed from that environment. And we have some threads including a FAQ article that discuss SFU.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. OS X (Apple)

Search recursive

before posting, I have tried to find my answer elsewhere. no luck. I need to find a file buried in a folder somewhere. Master folder has 10 sub folders. each sub folder has folders too. I found this but it does nothing I am on Mac and use Applescript. do shell script "find... (2 Replies)
Discussion started by: sbrady
2 Replies

2. UNIX for Dummies Questions & Answers

recursive search and ftp

Could someone help me in recursive search and ftp'ing the files to remote server? The host machine will have /dir1/dira/list_of_files1 /dir1/dirb/list_of_files2 /dir1/dirc/list_of_files3 . . . so., I need to search from dir1 recursively (only one level down) and find all the files that... (1 Reply)
Discussion started by: brahmi
1 Replies

3. UNIX for Dummies Questions & Answers

Recursive Permissions???

Is there anyway that I can change permissions on a directory and all its sub-directories and files using one single "chmod" command?? (5 Replies)
Discussion started by: the_red_dove
5 Replies

4. UNIX for Dummies Questions & Answers

recursive wc on a directory?

Hi all, I need to count the number of lines in all the files under a directory (several levels deep). I am feeling extremely dumb, but I don't know how to do that. Needless to say, I am not a shell script wiz... Any advice? thanks in advance! (13 Replies)
Discussion started by: bimba17
13 Replies

5. Cybersecurity

Recursive SFTP

Hello, I need to transfer files from Serve1 to Server2. Previously I was using scp command. Now I have to use sftp (due to audit issues). The problem with sftp is (atleast to my level of knowledge) we cannot transfer dirs (and files within that dir). Is there a way to solve this? Looks like... (1 Reply)
Discussion started by: MohanTJ
1 Replies

6. UNIX for Advanced & Expert Users

recursive sorting

In the ls command, -t option and -R option dont work simultaneously. ls -t ---> lists the files with sorting based on file date ls -R ---> lists the files recursively. How to make utilize both in the same command.? I want to sort the recursive files listing.. (1 Reply)
Discussion started by: fermisoft
1 Replies

7. Shell Programming and Scripting

recursive rcp

I wrote a shell script (AIX) to extract the file "/rep1/toto" from all the hosts referred in a list and send them to one local directory named ~/$host-$file with the hostname as prefix rcp -p user@host:/rep1/$file ~/$host-$file where file = toto ==> it works ! I would do the same thing... (6 Replies)
Discussion started by: Nicol
6 Replies

8. Shell Programming and Scripting

perl + Net::FTP::Recursive

Problem: It will not advance to the next user in the list. It always dies right after it sends the 2/2 files from the first users dir. $USERLIST="/export/home/mxdooley/perl_ftp/userlist"; $USER_DIR="/export/home/mxdooley/perl_ftp/homes";... (2 Replies)
Discussion started by: Optimus_P
2 Replies

9. UNIX for Dummies Questions & Answers

recursive effect!!

I run the following command in some of my folders... and ended up with a huge mess!! find . -type f -exec perl -e 's/blabla/zzzxxxx/gi' -p -i.bak {} \; I had to kill the process and later when I checked with one of my folders.. ls vaditerm.dt.bak vaditerm.dt.bak.bak... (2 Replies)
Discussion started by: sskb
2 Replies

10. UNIX for Dummies Questions & Answers

Recursive FTP

I am trying to write a recursive FTP script and have come to a point where I need to test if the file is either a normal ascii file or a directory. My question is how do I test if the file is either ascii or directory. (1 Reply)
Discussion started by: aslamg
1 Replies
Login or Register to Ask a Question