11-30-2006
Using the info gathered from this thread, this is what I have so far..
Code :
#! /usr/bin/ksh
#
# Goran Cvetanoski - 07/11/2006
#
# /export/ftpdir/autoftp
#
# This script uses ftp to move files from one server to another.
# It copies files from one servers Outgoing to another servers Incoming
# It FTPs to Incoming/.receiving directory and then moves the files to Incoming
# Once the files have been sent it moves the files from Outgoing to Outgoing/.sent
# It deletes files older then 14 days from both servers Outgoing/.sent directories
# It traps any FTP errors and send an email with the error message
# It can also send emails on successfull transfers, if wanted
#
# Available flags:
# -------------------------------------------------------------------------
# mget - ftp mget from $RHOST:\$ROUT to $LIN
# mput - ftp mput from $LOUT to $RHOST:\$RIN
# xfer - Perform a mget then an mput on $RHOST
#
# Two log files are created:
# -------------------------------------------------------------------------
# ftp.mail - This gets emailed to $ADMIN , if comment removed
# ftp.log - Verbose output from the ftp
#
# -------------------------------------------------------------------------
# A small note on all the fd stuff..
# -------------------------------------------------------------------------
# exec 4>&1
# ftp -nv >&4 2>&1 |&
#
# fd 0 is standard-in, fd 1 is standard-out, and fd 2 is standard-error.
# The line "exec 4&>1" opens fd 4 and assigns it to whatever fd 1 was
# assigned to. I am sorta "saving a copy of fd 1 in fd 4".
#
# The line "ftp -nv >&4 2>&1 |&" is a little harder.
#
# The "|&" turns the process into a co-process that allows subsequent
# "print -p" statements to send lines to the co-process' standard-in and
# "read -p" to read from its standard-out. So ksh forks a copy of itself
# and fiddles with the fd's 0 and 1 until this it set-up. But it leaves
# the rest of the fd's alone.
#
# Then it encounters ">&4" which causes it to set the ftp process' standard
# out to whatever 4 is. Well since 4 is a copy of 1 before the co-process,
# we are back to writing to the original shell's standard out. Lastly,
# the 2>&1 does the same thing for standard error.
#
# CHANGE LOG
# =========================================================================
# 07/11/2006 - Goran Base script created
# 09/11/2006 - Goran Created usage function
# 24/11/2006 - Goran Added error checking of ftp transfers
# 27/11/2006 - Goran Added ftptest function
# 01/12/2006 - Goran Added cleanup function
# 25/10/2007 - Goran Added CheckDir function
#
LHOST=`uname -n`
RHOST=123.456.789.123
USER=Enter your userid here
PASSWD=Enter your password here
ADMIN=unix-admin@mydomain.com
TSTFILE=TEST.TXT
LOG=/var/adm/ftp.log
LOG1=/tmp/ftp.mail
LOG2=/tmp/ftp.log
LOG3=/tmp/ftp.parselog
LIN=/export/ftpdir/Incoming
LOUT=/export/ftpdir/Outgoing
RIN=/$USER/Incoming
ROUT=/$USER/Outgoing
usage ()
{
echo ""
echo ""
echo " #########################################################"
echo " ## ##"
echo " ###### Usage: autoftp [ mget | mput | xfer ] ######"
echo " ## ##"
echo " #########################################################"
echo " ## ##"
echo " ## mget: Copy files from $RHOST to $LHOST ##"
echo " ## ##"
echo " ## mput: Copy files from $LHOST to $RHOST ##"
echo " ## ##"
echo " ## xfer: Perform an mget and an mput from $LHOST ##"
echo " ## to $RHOST ##"
echo " ## ##"
echo " #########################################################"
echo ""
echo ""
}
SendMail()
{
SUBJECT=$1
OUTPUT=$2
cat $OUTPUT | mailx -s "$SUBJECT" $ADMIN
}
CheckDir()
{
DIR=$1
if [ ! -d $DIR ]; then
echo "Error - Directory $DIR does not exist" >> $LOG2
echo "ERROR: Directory Error - $DIR does not exist" | mailx -s "Directory Error - $DIR" $ADMIN
exit 1
fi
}
doftp ()
{
CMD=$1
LOCAL=$2
REMOTE=$3
if [[ $CMD = "mget" ]]; then
echo "================================================" >> $LOG2
echo "== Receiving Files From $RHOST ==" >> $LOG2
echo "================================================" >> $LOG2
else
echo "============================================" >> $LOG2
echo "== Sending Files to $RHOST ==" >> $LOG2
echo "============================================" >> $LOG2
fi
exec 4>&1
ftp -nv >&4 2>&1 |&
pid2=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p lcd $LOCAL
print -p cd $REMOTE
print -p prompt
print -p $CMD \*
print -p bye
wait $pid2
}
ftptest ()
{
mv $LIN/$TSTFILE $LOUT
echo "=======================================" >> $LOG2
echo "== $TSTFILE being sent back to $RHOST ==" >> $LOG2
echo "=======================================" >> $LOG2
exec 4>&1
ftp -nv >&4 2>&1 |&
pid4=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p lcd $LOUT
print -p cd $RIN/.receiving
print -p put $TSTFILE
print -p bye
wait $pid4
}
log ()
{
CMD=$1
SRC=$2
echo "=== $CMD Files Report ===" >> $LOG1
cd $SRC
echo "Size(KB) File" >> $LOG1
ls -s | grep -v total | awk '{ $size = $1 / 2; print $size " " $2 }' | sort -n >> $LOG1
echo "=============================" >> $LOG1
}
mvfiles ()
{
LOCAL=$1
LDEST=$2
REMOTE=$3
RDEST=$4
exec 4>&1
ftp -n >&4 2>&1 |&
pid3=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p cd $REMOTE
cd $LOCAL
for files in `ls` ; do
print -p rename $files $RDEST/$files
mv $files $LDEST
done
print -p bye
wait $pid3
}
chkerror ()
{
cat /dev/null > $LOG3
cat $LOG2 >> $LOG
grep -v bytes $LOG2 >> $LOG3
for error in `awk '{print $1}' $LOG3`; do
case $error in
Not) SendMail "FTP Error - between $LHOST and $RHOST" $LOG2
exit 1;;
332) SendMail "FTP Error - Need account for login" $LOG2
exit 1;;
350) SendMail "FTP Error - Requested file action pending further information" $LOG2
exit 1;;
421) SendMail "FTP Error - Service not available, closing control connection" $LOG2
#This may be a reply to any command if the service knows it
#must shut down.
exit 1;;
425) SendMail "FTP Error - Can't open data connection" $LOG2
exit 1;;
426) SendMail "FTP Error - Connection closed; transfer aborted" $LOG2
exit 1;;
450) SendMail "FTP Error - Requested file action not taken" $LOG2
#File unavailable (e.g., file busy).
exit 1;;
451) SendMail "FTP Error - Requested action aborted: local error in processing" $LOG2
exit 1;;
452) SendMail "FTP Error - Requested action not taken" $LOG2
#Insufficient storage space in system.
exit 1;;
500) SendMail "FTP Error - Syntax error, command unrecognized" $LOG2
#This may include errors such as command line too long.
exit 1;;
501) SendMail "FTP Error - Syntax error in parameters or arguments" $LOG2
exit 1;;
502) SendMail "FTP Error - Command not implemented" $LOG2
exit 1;;
503) SendMail "FTP Error - Bad sequence of commands" $LOG2
exit 1;;
504) SendMail "FTP Error - Command not implemented for that parameter" $LOG2
exit 1;;
530) SendMail "FTP Error - Not logged in" $LOG2
exit 1;;
532) SendMail "FTP Error - Need account for storing files" $LOG2
exit 1;;
550) SendMail "FTP Error - No such file or directory" $LOG2
#File unavailable (e.g., directory empty, no access).
exit 1;;
551) SendMail "FTP Error - Requested action aborted: page type unknown" $LOG2
exit 1;;
552) SendMail "FTP Error - Requested file action aborted" $LOG2
#Exceeded storage allocation (for current directory or
#dataset).
exit 1;;
553) SendMail "FTP Error - Requested action not taken - File name not allowed" $LOG2
exit 1;;
esac
done
}
cleanup ()
{
echo "=======================================" >> $LOG1
echo "=======================================" >> $LOG2
echo "== Cleaning up old sent files ==" >> $LOG1
echo "== Cleaning up old sent files ==" >> $LOG2
echo "=======================================" >> $LOG1
echo "=======================================" >> $LOG2
echo " Files removed from $LHOST:$LOUT/.sent" >> $LOG1
echo " Files removed from $LHOST:$LOUT/.sent" >> $LOG2
## Clean up local servers Outgoing folder
cd $LOUT/.sent
find $LOUT/.sent -mtime +14 >> $LOG1
find $LOUT/.sent -mtime +14 >> $LOG2
find $LOUT/.sent -mtime +14 -exec rm {} \;
## Create the .history folder
cat /dev/null > $LIN/.history/.list
cd $LIN
for history in `ls` ; do
touch $LIN/.history/$history
done
exec 4>&1
ftp -n >&4 2>&1 |&
pid5=$!
print -p open $RHOST
print -p user $USER $PASSWD
print -p binary
print -p cd $ROUT/.sent
cd $LIN/.history
find $LIN/.history -mtime +14 >> $LIN/.history/.list
for old in `cat $LIN/.history/.list` ; do
print -p delete $old # Delete file from remote server
rm $old # Delete file from local folder
done
print -p bye
wait $pid5
echo " Files removed from $RHOST:$ROUT/.sent" >> $LOG1
echo " Files removed from $RHOST:$ROUT/.sent" >> $LOG2
cat $LIN/.history/.list >> $LOG1
cat $LIN/.history/.list >> $LOG2
echo "" >> $LOG2
}
## Create log headers here
echo "=============================" > $LOG1
echo "=============================" > $LOG2
echo " `date`" >> $LOG1
echo " `date`" >> $LOG2
echo "=============================" >> $LOG1
echo "=============================" >> $LOG2
CheckDir $BASE
CheckDir $LIN
CheckDir $LOUT
## Create required local directories
test ! -d $LIN/.receiving && mkdir $LIN/.receiving
test ! -d $LIN/.history && mkdir $LIN/.history
test ! -d $LOUT/.sent && mkdir $LOUT/.sent
CheckDir $LIN/.receiving
CheckDir $LIN/.history
CheckDir $LOUT/.sent
exec 5>&1
# Switch the comments on the next 2 lines if
# you also want to see output on the screen
#tee /dev/tty >> $LOG2 |&
tee >> $LOG2 |&
pid1=$!
exec >&p 2>&1 4>&1
case $1 in
mget) doftp mget $LIN/.receiving $ROUT
log Recieved $LIN/.receiving
mvfiles $LIN/.receiving .. $ROUT .sent ;;
mput) doftp mput $LOUT $RIN/.receiving
log Sent $LOUT
mvfiles $LOUT .sent $RIN/.receiving .. ;;
xfer) doftp mget $LIN/.receiving $ROUT
log Recieved $LIN/.receiving
mvfiles $LIN/.receiving .. $ROUT .sent
doftp mput $LOUT $RIN/.receiving
log Sent $LOUT
mvfiles $LOUT .sent $RIN/.receiving .. ;;
*) usage
exec >&- 2>&- 4>&- >&5 2>&1
exit 1 ;;
esac
## Run ftptest if test file exists in remote Outgoing folder
if [[ -e "$LIN/$TSTFILE" ]]; then
ftptest $tstfile
mvfiles $LOUT .sent $RIN/.receiving ..
exec >&- 2>&- 4>&- >&5 2>&1
chkerror
SendMail "FTP Test Results - between $LHOST and $RHOST" $LOG2
exit 0
fi
exec >&- 2>&- 4>&- >&5 2>&1
## Remove sent files older then 14 days
## From local and remote servers
cleanup
chkerror
#Remove comment below to receive an email on successfull ftp
#SendMail "FTP Results - between $LHOST and $RHOST" $LOG1
exit 0
This is writen for a Solaris 8 system, thats why I have to do the ls -s and awk stuff and can't just use ls -h in my log function. I also need to remove all files older then 14 days from the .sent directory. This is easy on the server as I just use find, but its a bit more tricky on the remote Windows NT system.
UPDATED
I have updated the code above to include my cleanup function..
Last edited by Tornado; 10-08-2007 at 07:21 PM ..
9 More Discussions You Might Find Interesting
1. UNIX for Dummies Questions & Answers
If I wanted a machine to put a specific file onto another OS far across the internet via FTP - and I wanted to do it automatically not user intervented, how would I do that?
Use the PUT command?
The file name and position never changes, it gets overwritten and the system on the other end... (6 Replies)
Discussion started by: n9ninchd
6 Replies
2. UNIX for Advanced & Expert Users
My requiremnet is to write a FTP script which will ftp a file to a specified ftp url.
How can I automate the process usnig the unix cron. (11 Replies)
Discussion started by: shauche
11 Replies
3. UNIX for Dummies Questions & Answers
How can an automated script be setup to run at a later time in the day. cron runs recurring tasks. I am interested in a one time process.
I want to back up a number of files whenever I make changes to an application and ftp for backup purposes. The script to automate the zipping of files has... (1 Reply)
Discussion started by: msmkeml
1 Replies
4. IP Networking
I have seen the script posted yesterday for automated ftp
Can we do some thing like ftp ing multiple files in one script
Example input.txt has all files names to be ftped
input.txt ------
a.tar
b.ccp
c.perl
i need to ftp all the files present in input.txt
i tried something like... (0 Replies)
Discussion started by: pbsrinivas
0 Replies
5. Shell Programming and Scripting
Hi
I am trying to delete some specific files ( files other than created today) from the server on a cron basis. I wrote a small script, but I am stuck up in how to delete only specific files.
#!/usr/bin/expect -f
set IP
set timeout -1
spawn ftp $IP
expect ):
send "username\n"... (10 Replies)
Discussion started by: sangfroid
10 Replies
6. Shell Programming and Scripting
Automated FTP.
hint : use 'atd' to schedule to run a specific script.
An Env Variable should be created,say CONF_DIR which points to some dir.
@ some time, create a TAR file of this and FTP it to some server. (3 Replies)
Discussion started by: vishalzone2002
3 Replies
7. Shell Programming and Scripting
I want to do Automated FTP from linux client to LINUX server using a shell script after every one hour. And copies last most updated file from Linux server.
OR
You can say that whenever files get modified on LINUX server so it also copies on my LINUX client.
Linux SERVER path=... (4 Replies)
Discussion started by: wakhan
4 Replies
8. Shell Programming and Scripting
Hi all,
I'm using the following script to automated ftp files to 1 ftp servers
host=192.168.0.1
/usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp
open $host
bin
cd ${directory}
put $files
quit
ftp
and the .netrc file contain
machine 192.168.0.1
login abc... (4 Replies)
Discussion started by: varu0612
4 Replies
9. Shell Programming and Scripting
Hello,I just know the basics of scripting & I need a script to automate ftp since I use it quite often.
I use ftp to transfer different kind of files and everytime the source & the destination directories are different for transferring files, so can anyone help me out on this urgently.. (2 Replies)
Discussion started by: mohit_02mar
2 Replies