Automated FTP task


 
Thread Tools Search this Thread
Special Forums IP Networking Automated FTP task
# 1  
Old 08-29-2001
Question Automated FTP task

Every day i ftp tar.gz a file from the production server to a back up machine.. This task creates way to much traffic on the network at the end of the day and puts and undo load on the production machine during operation hours. i would like to create a script that would automatically fire off the ftp at like 2:00 am so that the transfer occurs when there is little or no load on the network. if anyone has suggestions on how to accomplish this, PLEASE let me know...

thanks..

--e0

loworderbit@aol.com
# 2  
Old 08-29-2001
auto-ftp transfers

You can set up a ".netrc" file for the
user doing the transfer then set up a cron
job for that userid to run a script to actually
do the FTP transfer.
Check the man pages for ftp(1) and netrc(5)
# 3  
Old 08-29-2001
I need to do a lot of automated ftp jobs myself. I have tried several versions of this and have finally settled on a style of script that I like. I found that using a .netrc file to automate the logging-in process kept painting me into a corner because different scripts needed to sign in as different users. So I avoid .netrc and force the script to sign in. I don't like to allocate pty's unless I really am forced into it, so I also avoid pty based tools like expect. I really like ksh so that was my tool of choice. And I like the co-process concept because it makes feeding commands into the ftp process so easy with "print -p". The only problem is that the co-process manipulates standard-out so as to make it available to "read -p". And it's too hard to know how many "read -p" I will need. So I send the output to a different file descriptor. Putting it all together:

Code:
#! /usr/bin/ksh

HOST=remote.host.name
USER=whoever
PASSWD=whatever

exec 4>&1
ftp -nv >&4 2>&4 |&

print -p open $HOST
print -p user $USER $PASSWD
print -p cd directory
print -p binary
print -p put tar.gz
print -p bye

wait
exit 0

That script will tranfer the file and the output of the script will be the output from the ftp job itself. Put the script into cron and save the output so you can look at it the next morning.

Last edited by Perderabo; 07-02-2004 at 09:44 AM..
# 4  
Old 08-29-2001
hmm... write a script on the server that is launched daily using the cron (man crontab)... have that script ftp to your workstation or wherever and grab the file. The automated FTP will probably require a .netrc file in your user directory or wherever you ftp to to get the file.

However, if you are having to tar the file manually every day, write a script on the workstation side to do that for you a few minutes before the server script executes... once again, utilizing the crontab.

Good luck and let me know how it goes.

Strange... those other replies both popped up while I was writing... ummm... take their advice. Smilie

Last edited by ober5861; 08-29-2001 at 03:40 PM..
# 5  
Old 08-29-2001
Quote:
exec 4>&1
ftp -nv >&4 2>&4 |&
Perderabo... can you explain what that does exactly?

# 6  
Old 08-29-2001
Quote:
Originally posted by ober5861


Perderabo... can you explain what that does exactly?

That is the file descriptor manipulation. Recall that fd 0 is standard-in, fd 1 is standard-out, and fd 2 is standard-error. The line "exec 4&>1" opens fd 4 and assigns it to whatever fd 1 was assigned to. As you will see, I am sorta "saving a copy of fd 1 in fd 4".

The line "ftp -nv >&4 2>&1 |&" is a little harder.

The "|&" turns the process into a co-process that allows subsequent "print -p" statements to send lines to the co-process' standard-in and "read -p" to read from its standard-out. So ksh forks a copy of itself and fiddles with the fd's 0 and 1 until this it set-up. But it leaves the rest of the fd's alone.

Then it encounters ">&4" which causes it to set the ftp process' standard out to whatever 4 is. Well since 4 is a copy of 1 before the co-process, we are back to writing to the original shell's standard out. Lastly, the 2>&4 does the same thing for standard error. I could've used "2>&1" at this point for the same effect.

This is hard to explain, but I hope this helps.

# 7  
Old 08-29-2001
Ftp

thanks for all of the help.. i guess i should have mentioned that i am running sco openserver.. i dont know if it matters or not, but the more information the better.... anyway, it looks like the best way to do this is to set up a script and use crontab..
manually, i was using the get command from the back up machine, but if its all the same, put from the production machine will work fine.
now, its time to get down to the creation of the script and setting up cron to run it...
my unix background is about three weeks, but i have been an (i apologize in advanced) NT admin for over a year, so please forgive my ignorance.
anyway, if any of you would like to assist me in setting this up i would be very gratefull... This website, Unix: the complete reference, and sco unix in a nutshell are the only resources i have so far, so i am relying on you all a little...

thanks again...

e0--
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Automated FTP

Hello,I just know the basics of scripting & I need a script to automate ftp since I use it quite often. I use ftp to transfer different kind of files and everytime the source & the destination directories are different for transferring files, so can anyone help me out on this urgently.. (2 Replies)
Discussion started by: mohit_02mar
2 Replies

2. Shell Programming and Scripting

Automated FTP script using .netrc to multiple FTP servers

Hi all, I'm using the following script to automated ftp files to 1 ftp servers host=192.168.0.1 /usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp open $host bin cd ${directory} put $files quit ftp and the .netrc file contain machine 192.168.0.1 login abc... (4 Replies)
Discussion started by: varu0612
4 Replies

3. Shell Programming and Scripting

Automated FTP

I want to do Automated FTP from linux client to LINUX server using a shell script after every one hour. And copies last most updated file from Linux server. OR You can say that whenever files get modified on LINUX server so it also copies on my LINUX client. Linux SERVER path=... (4 Replies)
Discussion started by: wakhan
4 Replies

4. Shell Programming and Scripting

how to devlop automated FTP in UNIX

Automated FTP. hint : use 'atd' to schedule to run a specific script. An Env Variable should be created,say CONF_DIR which points to some dir. @ some time, create a TAR file of this and FTP it to some server. (3 Replies)
Discussion started by: vishalzone2002
3 Replies

5. Shell Programming and Scripting

automated ftp.

Hi I am trying to delete some specific files ( files other than created today) from the server on a cron basis. I wrote a small script, but I am stuck up in how to delete only specific files. #!/usr/bin/expect -f set IP set timeout -1 spawn ftp $IP expect ): send "username\n"... (10 Replies)
Discussion started by: sangfroid
10 Replies

6. IP Networking

Automated ftp for Multiple files

I have seen the script posted yesterday for automated ftp Can we do some thing like ftp ing multiple files in one script Example input.txt has all files names to be ftped input.txt ------ a.tar b.ccp c.perl i need to ftp all the files present in input.txt i tried something like... (0 Replies)
Discussion started by: pbsrinivas
0 Replies

7. UNIX for Dummies Questions & Answers

one time automated ftp session

How can an automated script be setup to run at a later time in the day. cron runs recurring tasks. I am interested in a one time process. I want to back up a number of files whenever I make changes to an application and ftp for backup purposes. The script to automate the zipping of files has... (1 Reply)
Discussion started by: msmkeml
1 Replies

8. UNIX for Advanced & Expert Users

Automated FTP

My requiremnet is to write a FTP script which will ftp a file to a specified ftp url. How can I automate the process usnig the unix cron. (11 Replies)
Discussion started by: shauche
11 Replies

9. UNIX for Dummies Questions & Answers

FTP automated?

If I wanted a machine to put a specific file onto another OS far across the internet via FTP - and I wanted to do it automatically not user intervented, how would I do that? Use the PUT command? The file name and position never changes, it gets overwritten and the system on the other end... (6 Replies)
Discussion started by: n9ninchd
6 Replies
Login or Register to Ask a Question