FTP Script Help!!


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FTP Script Help!!
# 22  
Old 11-05-2014
I have seen some scripts that put the file over with a temporary name then rename. This also protects from jobs on the destination server attempting to process file still in transit.

Once the remote rename is done the file can then be moved to a transfered folder, for eaxample:

Code:
put ORD049382.20141106 tmp_transfer
rename tmp_transfer ORD49382.20141106
!mv ORD049382.20141106 tmp /var/orders/transfered/


Here is an attempt at putting Don Cragun's suggestions and this change all together for you:

Code:
#!/bin/bash
source /opt/foobar/$1/config/usrconfig.sh
source  /opt/foobar/$ENV/config/$2

### foobar environment variables ###
export FTPHOST="sftpserv.foobar.com"                 # server to ftp to
export FTPUSR="usmssh"                               # ftp user id
export FTPPASS=""                                    # ftp user password
export PATTERN=                                      # pattern for file name to ftp
export USING_MARKER=no                               # using a marker file???
#export TDEMARKER="X"                                # pattern for marker file if no files to ftp
export FTPDIR="prod/TDE/in/"  # directory to ftp to on ftp server
export DAEMONLOG=/opt/foobar/$ENV/log/TDEdeamon.log
export DAEMONSLEEP=10
export CURRDATE=`date +%Y%m%d`
export READDIR=/opt/foobar/$ENV/data/TDE/output
export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=$APP_ROOT/data/TDE/backup/

if [ ! -d "$READDIR" ]
then
    echo "Directory $READDIR not found!" >>$DAEMONLOG 2>&1
    exit 1
fi

cd $READDIR
sleep 2

> tde_batch
find $READDIR -type f -name '*.ready' | while read file
do
  real_file="${file%.ready}"
  echo "put \"$real_file\" \"$FTPDIR/trans.temp\"" >> tde_batch
  echo "rename \"$FTPDIR/trans.temp\" \"$FTPDIR/$real_file\"" >> tde_batch
  echo "!mv \"$real_file\" \"$TDE_BACKUP/\"" >> tde_batch
done

if [ ! -s ./tde_batch ]
then
     echo "No ready files found in $READDIR" >>$DAEMONLOG 2>&1
     exit 2
fi

echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST" >>$DAEMONLOG 2>&1
mv tde_batch "$TDE_BACKUP/tde_batch.$CURRDATE"
exit 0

This User Gave Thanks to Chubler_XL For This Post:
# 23  
Old 11-05-2014
Quote:
Originally Posted by Don Cragun
With the code being discussed in this thread:
Code:
...
export READDIR=/home/foobaruser/TDE
#export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=/home/foobaruser/TDE/backup

cd $READDIR
...

as I said before, there is absolutely no way that the cd command shown in red above is going to "silently change directory to your $HOME if it can not use $READDIR to change". I said there were three ways that a cd command could move you to $HOME AND NONE of them are possible with this code.

I fully agree that ignoring exit codes from cd, cp, sftp, and other utilities is a recipe for disaster; but there is no way the above code is ever going to silently change directory to $HOME.
I don't see that first red highlighted code in my post, in question. Maybe there was some reason for not being included.

Quote:
Quote:
Originally Posted by gkelly1117
cd $READDIR
find $READDIR -type f -name '*.ready' | while read file
do
real_file="${file%.ready}"
echo "put $real_file $FTPDIR" >> tde_batch
cp $real_file $TDE_BACKUP
done
echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST"
cp tde_batch $TDE_BACKUP/tde_batch.$CURRDATE
rm tde_batch
rm *
exit 0[/CODE]
I would like to bring to your attention the following combination of commands in red, which has the potential of spread disaster.
In a script, cd into a directory and accepting it blindly, is very dangerous, for example:

The command cd will silently change directory to your $HOME if it can not use $READDIR to change due to being unset or empty. Once's in there I don't know the mayhem that it would do, but for sure you can kiss goodbye to your data in $HOME.

Some ways to minimize the risk is to use absolute paths and if you cd check $PWD or pwd, and compare with where you should be.

Furthermore, the content of the directory where you are executing the script from is at risk of being destroy or damaged, if $READDIR is set to a path that can not change into. Success of
Code:
if [ -d $READDIR ]

is not indication that you can cd into it, even if the directory exist.
In order to further clarify my intent at the first try, I would like to amend my original text with added blue.

Last edited by Aia; 11-05-2014 at 11:26 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

configuration for ftp script in main script

Hi, I am new to shell scripting,and i was planning to write a script that will FTP files to destination folder. I was planning that All configuration should be done through a properties files. and finally the output should be Files are transferred I have developed a properties file named... (4 Replies)
Discussion started by: rahul125
4 Replies

2. Shell Programming and Scripting

Need help in ftp script

hello , I am working on a FTP script which navigates to specific directory and pulls the file Issue which i am facing :: ---------------------------- i) When there is a file it pulls the file , but when there is not file in the directory it fails in the same statement and it is not... (3 Replies)
Discussion started by: ranga27
3 Replies

3. Linux

Need SCP script equivalent to FTP script .

Hi, Currently i'm using the folllowing FTP acript and its working. echo "open $server" > ftp_file echo "user $user $password" >> ftp_file echo "cd $remote_dir" >> ftp_file echo "lcd $local_dir" >> ftp_file echo "put $file">> ftp_file echo "bye" >> ftp_file ftp -nv < ftp_file I've... (1 Reply)
Discussion started by: vickramshetty
1 Replies

4. Shell Programming and Scripting

Automated FTP script using .netrc to multiple FTP servers

Hi all, I'm using the following script to automated ftp files to 1 ftp servers host=192.168.0.1 /usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp open $host bin cd ${directory} put $files quit ftp and the .netrc file contain machine 192.168.0.1 login abc... (4 Replies)
Discussion started by: varu0612
4 Replies

5. Shell Programming and Scripting

passing parameter to ftp script from output of another ftp

Hi, I have a ftp script which first gets all the file names and echo's the latest file. I'm using another ftp command sets to get the file name given by first ftp. The problem is the parameter is not accepted by second ftp. The error message i'm getting is > Rename Temp File calloc:ICMP... (5 Replies)
Discussion started by: ammu
5 Replies

6. Shell Programming and Scripting

ftp script not able to connect to ftp server.

I have the following ftp script to get files from a remote location. However, on running the script I find that I am not even able to connect to ftp server. I am able to connect to ftp server using other GUI ftp tools like WS_FTP using the same IP. IP used here is a dummy IP. What can go... (3 Replies)
Discussion started by: gram77
3 Replies

7. Shell Programming and Scripting

FTP script to FTP file to UNIX - Solaris

Hello, A couple of times per week, i receive emails notifications when files are available for processing. Currently i read these eamails with a java program and store the attachement on my C: drive and would now like to generate a PC script to send this file name up to UNIX-Solaris and... (3 Replies)
Discussion started by: bobk544
3 Replies

8. Shell Programming and Scripting

Need help - script for ftp..

I have my ftp script as below.. This is logging the messages into ftp.log file , But I want the same output(what ever messages are going into ftp.log) to be printed on the console too for the user to show the status.. Tried with "tee" .. unable to get the solution.. Can some one help me... (3 Replies)
Discussion started by: Srini75
3 Replies

9. UNIX for Dummies Questions & Answers

excuting a shell script within ftp script

Novice here... I need help with excuting a shell script on a flat file that I've transfered over from a Windows XP machine for manipulation through an auto FTP script... so that after it is transfers it excutes the shell script and then returns it back to XP machine... Any ideas... (2 Replies)
Discussion started by: Gerry405
2 Replies

10. Shell Programming and Scripting

FTP script in Unix shell script

Hello , I am trying to make a shell script (Unix) for a ftp connection to another server and to get a file. So I have no knowledge about ftp and my script must do automaticly the connection with the user and passwd. Can you help us about that... Christian... (2 Replies)
Discussion started by: steiner
2 Replies
Login or Register to Ask a Question