FTP Script Help!!


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FTP Script Help!!
# 1  
Old 10-31-2014
FTP Script Help!!

I am trying to write a script that allows me to check a directory for files that i will sftp to another server.

When the file is created it creates a .ready file that is supposed to signal a file is fully created and ready to sftp. for example a file named 10312014.13.07.40.27711113 would have trailing 10312014.13.07.40.27711113.ready file as well.

i dont know how to properly check for 10312014.13.07.40.27711113.ready and if it's there sftp 10312014.13.07.40.27711113.

below is what I have so far, it's just a shell of an idea, I'm stuck and any help would be greatly appreciated.

Thanks!

Code:
#!/bin/bash
source /opt/foobar/$1/config/usrconfig.sh
source  /opt/foobar/$ENV/config/$2

### foobar environment variables ###
export FTPHOST="sftpserv.foobar.com"                                    # server to ftp to
export FTPUSR="usmssh"                                              # ftp user id
export FTPPASS=""                                      # ftp user password
export PATTERN=                                      # pattern for file name to ftp
export USING_MARKER=no                                          # using a marker file???
#export TDEMARKER="X"                                    # pattern for marker file if no files to ftp
export FTPDIR="prod/TDE/in/"  # directory to ftp to on ftp server
export DAEMONLOG=/opt/foobar/$ENV/log/TDEdeamon.log
export DAEMONSLEEP=10
export CURRDATE=`date +%Y%m%d`
export READDIR=/opt/foobar/$ENV/data/TDE/output
export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=$APP_ROOT/data/TDE/backup/

cd $READDIR
if [ -f  $READDIR ]
   then
     echo " $CURRDATE files found to send!"
   else
     echo "no files found in $READDIR to send!"
     exit
fi
sleep 2


for i in `ls $READDIR`
 do
        cp $i $TDE_BACKUP
        echo $i


        if [ -f $i.ready ]
        then
         rm *.ready

			/usr/bin/sftp $FTPUSR@$FTPHOST << E_O_F
			cd $FTPDIR
			put  $i
			ls -l $PATTERN*
			quit
E_O_F
        else
        echo "No files are ready"
        fi
done

.
# 2  
Old 10-31-2014
for I in `ls *` is a useless use of backticks. for I in * does the exact same thing with one fewer command and no unintended side-effects.

You can upload as the name you want with "put localfilename remotefilename".

You can feed the list of files into sftp by using a pipe, avoiding running sftp 537 times for 537 files.

Code:
#!/bin/sh

(
        cd $READDIR

        echo "cd $FTPDIR"

        for FILE in *.ready
        do
                [ -e "$FILE" ] || break # Avoids error
                echo "put ${FILE} $FTPDIR/${FILE/.ready/}"
        done

        echo "ls -l $PATTERN"
        echo "quit"
) | /usr/bin/sftp $FTPUSR@$FTPHOST

This just uploads the file, I wasn't quite sure what else you wanted done with them.

Last edited by Corona688; 10-31-2014 at 06:53 PM..
These 2 Users Gave Thanks to Corona688 For This Post:
# 3  
Old 11-01-2014
Quote:
Originally Posted by gkelly1117
. . .
i dont know how to properly check for 10312014.13.07.40.27711113.ready and if it's there sftp 10312014.13.07.40.27711113.
. . .
You'll need to slightly modify Corona688's proposal for this:
Code:
        for FR in *.ready 
          do FILE="${FILE/.ready/}" 
             [ -e "$FILE" ] || break # Avoids error
             echo "put ${FILE} $FTPDIR/${FILE}"

This will get all "*.ready" filenames, remove the ".ready" part, and sftp the actual file as desired. Is the dot at the end part of the filename or a full stop? If the former, adapt the expansion pattern.
These 2 Users Gave Thanks to RudiC For This Post:
# 4  
Old 11-01-2014
Quote:
Originally Posted by RudiC
You'll need to slightly modify Corona688's proposal for this:
Code:
        for FR in *.ready 
          do FILE="${FILE/.ready/}" 
             [ -e "$FILE" ] || break # Avoids error
             echo "put ${FILE} $FTPDIR/${FILE}"

This will get all "*.ready" filenames, remove the ".ready" part, and sftp the actual file as desired. Is the dot at the end part of the filename or a full stop? If the former, adapt the expansion pattern.
Almost. You used FILE where you needed FR:
Code:
        for FR in *.ready 
          do FILE="${FR%.ready}" 
             [ -e "$FILE" ] || break # Avoids error
             echo "put ${FILE} $FTPDIR/${FILE}"

I also switched from ${var/expr/rep} to ${var%pattern} since the latter is in the standards and is accepted by more shells.

Depending on how solid your file creator is, you might also want to change the break to a continue. Or to do what I think Corona688 was trying to do, change that line to:
Code:
             [ -e "$FR" ] || continue # Avoids error

which catches the case where there are no matching files and FR will be set to the literal string *.ready. (But, there isn't likely to be a flle named * either, so this probably doesn't matter.)
These 3 Users Gave Thanks to Don Cragun For This Post:
# 5  
Old 11-03-2014
Thanks so much guys for your help! your input helped me get over my first hurdle, however another one arose that I am having issues with, and I am sure it has to do with how I coded this thing up.

I need to send multiple files, not just one. I thought I could send multiple like this, but I cant, so I played with creating a batch file, but I am clearly not doing this right.

Please see below code.

What am I doing wrong now? Thanks in advanced!


Code:
#!/bin/bash -x
source /opt/foobar/$1/config/usrconfig.sh

### foobar environment variables ###
export FTPHOST="sftpserv.foobar.com"                                    # server to ftp to
export FTPUSR="foouser"                                              # ftp user id
export FTPPASS="foo123"                                      # ftp user password
export PATTERN=                                      # pattern for file name to ftp
export USING_MARKER=no                                          # using a marker file???
#export TDEMARKER="X"                                    # pattern for marker file if no files to ftp
export FTPDIR="/home/foo"  # directory to ftp to on ftp server
export DAEMONLOG=/home/foobaruser/TDEdeamon.log
export DAEMONSLEEP=10
export CURRDATE=`date +%d-%m-%Y-%H-%M`
export READDIR=/home/foobaruser/TDE
#export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=/home/foobaruser/TDE/backup

cd $READDIR
if [ -a  $READDIR ]
   then
     echo " $CURRDATE files found to send in $READDIR!" >>$DAEMONLOG 2>&1
   else
     echo "no files found in $READDIR to send!" >>$DAEMONLOG 2>&1
     exit
fi
sleep 2

touch tde_batch
for i in *.ready
 do {
 FILE="${i%.ready}"
 [ -e "$FILE" ] || continue
  echo " Going to put ${FILE} in $FTPDIR directory" >>$DAEMONLOG 2>&1
  echo "put ${FILE}" >> tde_batch
  }
  echo "quit" >> tde_batch
  
  sftp -b tde_batch $FTPUSR@$FTPHOST  
  
        cp $i $TDE_BACKUP
            cd $TDE_BACKUP
            rm *.ready
done

# 6  
Old 11-03-2014
Moderator's Comments:
Mod Comment Posting "Does not work" without explanation does not help you or anyone. If a command does not work for you, please show the exact circumstances you used it, and the exact error or malfunction you received. Do not paraphrase errors, or post the text as links, images, or attachments if you can avoid it: Paste the exact message, in code tags, like [code] text [/code] or by selecting the text and using the Image button.

Thank you.

The UNIX and Linux Forums
# 7  
Old 11-03-2014
What errors do you get?

You might be safer to remove a single .ready files within in the loop so if any more arrive whilst you are processing, they don't get the flag removed.

I don't see why you have the curly brackets/braces in the loop. Can you explain? You are sending a single file each time. Is this intended? I'd be happy with that, but you complicate it with the wildcard rm *.ready




Robin
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

configuration for ftp script in main script

Hi, I am new to shell scripting,and i was planning to write a script that will FTP files to destination folder. I was planning that All configuration should be done through a properties files. and finally the output should be Files are transferred I have developed a properties file named... (4 Replies)
Discussion started by: rahul125
4 Replies

2. Shell Programming and Scripting

Need help in ftp script

hello , I am working on a FTP script which navigates to specific directory and pulls the file Issue which i am facing :: ---------------------------- i) When there is a file it pulls the file , but when there is not file in the directory it fails in the same statement and it is not... (3 Replies)
Discussion started by: ranga27
3 Replies

3. Linux

Need SCP script equivalent to FTP script .

Hi, Currently i'm using the folllowing FTP acript and its working. echo "open $server" > ftp_file echo "user $user $password" >> ftp_file echo "cd $remote_dir" >> ftp_file echo "lcd $local_dir" >> ftp_file echo "put $file">> ftp_file echo "bye" >> ftp_file ftp -nv < ftp_file I've... (1 Reply)
Discussion started by: vickramshetty
1 Replies

4. Shell Programming and Scripting

Automated FTP script using .netrc to multiple FTP servers

Hi all, I'm using the following script to automated ftp files to 1 ftp servers host=192.168.0.1 /usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp open $host bin cd ${directory} put $files quit ftp and the .netrc file contain machine 192.168.0.1 login abc... (4 Replies)
Discussion started by: varu0612
4 Replies

5. Shell Programming and Scripting

passing parameter to ftp script from output of another ftp

Hi, I have a ftp script which first gets all the file names and echo's the latest file. I'm using another ftp command sets to get the file name given by first ftp. The problem is the parameter is not accepted by second ftp. The error message i'm getting is > Rename Temp File calloc:ICMP... (5 Replies)
Discussion started by: ammu
5 Replies

6. Shell Programming and Scripting

ftp script not able to connect to ftp server.

I have the following ftp script to get files from a remote location. However, on running the script I find that I am not even able to connect to ftp server. I am able to connect to ftp server using other GUI ftp tools like WS_FTP using the same IP. IP used here is a dummy IP. What can go... (3 Replies)
Discussion started by: gram77
3 Replies

7. Shell Programming and Scripting

FTP script to FTP file to UNIX - Solaris

Hello, A couple of times per week, i receive emails notifications when files are available for processing. Currently i read these eamails with a java program and store the attachement on my C: drive and would now like to generate a PC script to send this file name up to UNIX-Solaris and... (3 Replies)
Discussion started by: bobk544
3 Replies

8. Shell Programming and Scripting

Need help - script for ftp..

I have my ftp script as below.. This is logging the messages into ftp.log file , But I want the same output(what ever messages are going into ftp.log) to be printed on the console too for the user to show the status.. Tried with "tee" .. unable to get the solution.. Can some one help me... (3 Replies)
Discussion started by: Srini75
3 Replies

9. UNIX for Dummies Questions & Answers

excuting a shell script within ftp script

Novice here... I need help with excuting a shell script on a flat file that I've transfered over from a Windows XP machine for manipulation through an auto FTP script... so that after it is transfers it excutes the shell script and then returns it back to XP machine... Any ideas... (2 Replies)
Discussion started by: Gerry405
2 Replies

10. Shell Programming and Scripting

FTP script in Unix shell script

Hello , I am trying to make a shell script (Unix) for a ftp connection to another server and to get a file. So I have no knowledge about ftp and my script must do automaticly the connection with the user and passwd. Can you help us about that... Christian... (2 Replies)
Discussion started by: steiner
2 Replies
Login or Register to Ask a Question