FTP Script Help!!


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting FTP Script Help!!
# 8  
Old 11-04-2014
Quote:
Originally Posted by Corona688
Moderator's Comments:
Mod Comment Posting "Does not work" without explanation does not help you or anyone. If a command does not work for you, please show the exact circumstances you used it, and the exact error or malfunction you received. Do not paraphrase errors, or post the text as links, images, or attachments if you can avoid it: Paste the exact message, in code tags, like [code] text [/code] or by selecting the text and using the Image button.

Thank you.

The UNIX and Linux Forums
Sorry about that, I will post proper errors that can be used

---------- Post updated at 12:14 PM ---------- Previous update was at 12:13 PM ----------

Quote:
Originally Posted by rbatte1
What errors do you get?

You might be safer to remove a single .ready files within in the loop so if any more arrive whilst you are processing, they don't get the flag removed.

I don't see why you have the curly brackets/braces in the loop. Can you explain? You are sending a single file each time. Is this intended? I'd be happy with that, but you complicate it with the wildcard rm *.ready




Robin
No I want to send multiple files if possible. as for the curly braces, I thought they would help my problem. No real reason other than that.

---------- Post updated at 12:19 PM ---------- Previous update was at 12:14 PM ----------

I ran the script with a -x below is the output.

It looks like it's running fine until it finds the first file, it echo's the file, writes it to the batch file, then echoes the quit to the batch file, I want to traverse, echo every single file into the batch file first, then echo the quit, then let the batch file sftp.

My logic is clearly wrong, some proper directive would be appreciated!

Thanks! I hope this is better information to help with. Also I see the permission denied error, but I assume that batch needs ssh keys shared and not a password, in the production env, the keys are already shared and that wont be an issue

Code:
+ source /opt/foobar/or_foo/config/usrconfig.sh
+ export FTPHOST=foobar01.foobar.com
+ FTPHOST=foobar01.foobar.com
+ export FTPUSR=foouser
+ FTPUSR=foouser
+ export FTPPASS=fooy123
+ FTPPASS=fooy123
+ export PATTERN=
+ PATTERN=
+ export USING_MARKER=no
+ USING_MARKER=no
+ export FTPDIR=/home/foouser
+ FTPDIR=/home/foouser
+ export DAEMONLOG=/home/foobaruser/TDEdeamon.log
+ DAEMONLOG=/home/foobaruser/TDEdeamon.log
+ export DAEMONSLEEP=10
+ DAEMONSLEEP=10
++ date +%d-%m-%Y-%H-%M
+ export CURRDATE=03-11-2014-10-07
+ CURRDATE=03-11-2014-10-07
+ export READDIR=/home/foobaruser/TDE
+ READDIR=/home/foobaruser/TDE
+ export TDE_BACKUP=/home/foobaruser/TDE/backup
+ TDE_BACKUP=/home/foobaruser/TDE/backup
+ cd /home/foobaruser/TDE
+ '[' -a /home/foobaruser/TDE ']'
+ echo ' 03-11-2014-10-07 files found to send in /home/foobaruser/TDE!'
+ sleep 2
+ touch tde_batch
+ for i in '*.ready'
+ FILE=10312014.10.36.51.52667612
+ '[' -e 10312014.10.36.51.52667612 ']'
+ echo ' Going to put 10312014.10.36.51.52667612 in /home/foouser directory'
+ echo 'put 10312014.10.36.51.52667612'
+ echo quit
+ sftp -b tde_batch foouser@foobar01.foobar.com
Permission denied (gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive).
Couldn't read packet: Connection reset by peer
+ cp 10312014.10.36.51.52667612.ready /home/foobaruser/TDE/backup
+ cd /home/foobaruser/TDE/backup
+ rm 10312014.10.36.51.52667612.ready
+ for i in '*.ready'
+ FILE=10312014.11.15.41.64934033
+ '[' -e 10312014.11.15.41.64934033 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.11.22.53.52444830
+ '[' -e 10312014.11.22.53.52444830 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.12.34.26.49837220
+ '[' -e 10312014.12.34.26.49837220 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.12.36.16.5815654
+ '[' -e 10312014.12.36.16.5815654 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.12.47.03.24215656
+ '[' -e 10312014.12.47.03.24215656 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.12.49.21.40757399
+ '[' -e 10312014.12.49.21.40757399 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.13.01.41.84187558
+ '[' -e 10312014.13.01.41.84187558 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.13.07.18.23453400
+ '[' -e 10312014.13.07.18.23453400 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.13.07.40.27711113
+ '[' -e 10312014.13.07.40.27711113 ']'
+ continue
+ for i in '*.ready'
+ FILE=10312014.13.26.27.08714191
+ '[' -e 10312014.13.26.27.08714191 ']'
+ continue



[foobaruser@foobar02 TDE]$ cat tde_batch
put 10312014.10.36.51.52667612
quit

---------- Post updated 11-04-14 at 10:31 AM ---------- Previous update was 11-03-14 at 12:19 PM ----------

I think I need gidance on how to properly loop the traversing of files being put into the batch file before echoing quit
This User Gave Thanks to gkelly1117 For This Post:
# 9  
Old 11-04-2014
It looks like the logic could do what you are after. What do you get if you just run the following on the command line?:-
Code:
sftp foouser@foobar01.foobar.com

It might just be that the authentication is missing or incomplete.

If you get a prompt asking if you want to accept a fingerprint, then accepting it will store the fingerprint (effectively the remote server public key) in the know_hosts file so it will not prompt again. That has been a problem I've seen before sorted by a manual test first.



Thanks, in advance,
Robin
# 10  
Old 11-04-2014
I got it to work finally.

Below is finished product.

Thanks for all involved!

Code:
#!/bin/bash -x
source /opt/foobar/$1/config/usrconfig.sh

### foobar environment variables ###
export FTPHOST="sftpserv.foobar.com"                                    # server to ftp to
export FTPUSR="foouser"                                              # ftp user id
export FTPPASS="foo123"                                      # ftp user password
export PATTERN=                                      # pattern for file name to ftp
export USING_MARKER=no                                          # using a marker file???
#export TDEMARKER="X"                                    # pattern for marker file if no files to ftp
export FTPDIR="/home/foo"  # directory to ftp to on ftp server
export DAEMONLOG=/home/foobaruser/TDEdeamon.log
export DAEMONSLEEP=10
export CURRDATE=`date +%d-%m-%Y-%H-%M`
export READDIR=/home/foobaruser/TDE
#export TEMP=/opt/foobar/$ENV/tmp
export TDE_BACKUP=/home/foobaruser/TDE/backup

cd $READDIR
if [ -a  $READDIR ]
   then
     echo " $CURRDATE files found to send in $READDIR!" >>$DAEMONLOG 2>&1
   else
     echo "no files found in $READDIR to send!" >>$DAEMONLOG 2>&1
     exit
fi
sleep 2

touch tde_batch
cd $READDIR
find $READDIR -type f -name '*.ready' | while read file
do
  real_file="${file%.ready}"
  echo "put $real_file $FTPDIR" >> tde_batch
  cp $real_file $TDE_BACKUP
done
echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST"
cp tde_batch $TDE_BACKUP/tde_batch.$CURRDATE
rm tde_batch  
rm *
exit 0

# 11  
Old 11-04-2014
Be careful of the rm * at the end of your script.

If any files are created while the sftp transfer is running they will be removed without being sent to sftpserv.foobar.com.
# 12  
Old 11-04-2014
Quote:
Originally Posted by gkelly1117
cd $READDIR
find $READDIR -type f -name '*.ready' | while read file
do
real_file="${file%.ready}"
echo "put $real_file $FTPDIR" >> tde_batch
cp $real_file $TDE_BACKUP
done
echo "quit" >> tde_batch
sftp -b tde_batch "$FTPUSR@$FTPHOST"
cp tde_batch $TDE_BACKUP/tde_batch.$CURRDATE
rm tde_batch
rm *
exit 0[/CODE]
I would like to bring to your attention the following combination of commands in red, which has the potential of spread disaster.

The command cd will silently change directory to your $HOME if it can not use $READDIR to change. Once's in there I don't know the mayhem that it would do, but for sure you can kiss goodbye to your data in $HOME.

Some ways to minimize the risk is to use absolute paths and if you cd check $PWD or pwd, and compare with where you should be.
# 13  
Old 11-04-2014
The rm * bothers me as well. Any suggestions would be greatly appreciated. I just want to delete the files after I have copied them to a back up directory and SFTP'd them over
# 14  
Old 11-04-2014
How about:

Code:
find $READDIR -type f -name '*.ready' | while read file
do
  real_file="${file%.ready}"
  echo "put $real_file $FTPDIR" >> tde_batch
  echo "!rm $real_file" >> tde_batch
  cp $real_file $TDE_BACKUP
done

This User Gave Thanks to Chubler_XL For This Post:
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

configuration for ftp script in main script

Hi, I am new to shell scripting,and i was planning to write a script that will FTP files to destination folder. I was planning that All configuration should be done through a properties files. and finally the output should be Files are transferred I have developed a properties file named... (4 Replies)
Discussion started by: rahul125
4 Replies

2. Shell Programming and Scripting

Need help in ftp script

hello , I am working on a FTP script which navigates to specific directory and pulls the file Issue which i am facing :: ---------------------------- i) When there is a file it pulls the file , but when there is not file in the directory it fails in the same statement and it is not... (3 Replies)
Discussion started by: ranga27
3 Replies

3. Linux

Need SCP script equivalent to FTP script .

Hi, Currently i'm using the folllowing FTP acript and its working. echo "open $server" > ftp_file echo "user $user $password" >> ftp_file echo "cd $remote_dir" >> ftp_file echo "lcd $local_dir" >> ftp_file echo "put $file">> ftp_file echo "bye" >> ftp_file ftp -nv < ftp_file I've... (1 Reply)
Discussion started by: vickramshetty
1 Replies

4. Shell Programming and Scripting

Automated FTP script using .netrc to multiple FTP servers

Hi all, I'm using the following script to automated ftp files to 1 ftp servers host=192.168.0.1 /usr/bin/ftp -vi >> $bkplog 2>&1 <<ftp open $host bin cd ${directory} put $files quit ftp and the .netrc file contain machine 192.168.0.1 login abc... (4 Replies)
Discussion started by: varu0612
4 Replies

5. Shell Programming and Scripting

passing parameter to ftp script from output of another ftp

Hi, I have a ftp script which first gets all the file names and echo's the latest file. I'm using another ftp command sets to get the file name given by first ftp. The problem is the parameter is not accepted by second ftp. The error message i'm getting is > Rename Temp File calloc:ICMP... (5 Replies)
Discussion started by: ammu
5 Replies

6. Shell Programming and Scripting

ftp script not able to connect to ftp server.

I have the following ftp script to get files from a remote location. However, on running the script I find that I am not even able to connect to ftp server. I am able to connect to ftp server using other GUI ftp tools like WS_FTP using the same IP. IP used here is a dummy IP. What can go... (3 Replies)
Discussion started by: gram77
3 Replies

7. Shell Programming and Scripting

FTP script to FTP file to UNIX - Solaris

Hello, A couple of times per week, i receive emails notifications when files are available for processing. Currently i read these eamails with a java program and store the attachement on my C: drive and would now like to generate a PC script to send this file name up to UNIX-Solaris and... (3 Replies)
Discussion started by: bobk544
3 Replies

8. Shell Programming and Scripting

Need help - script for ftp..

I have my ftp script as below.. This is logging the messages into ftp.log file , But I want the same output(what ever messages are going into ftp.log) to be printed on the console too for the user to show the status.. Tried with "tee" .. unable to get the solution.. Can some one help me... (3 Replies)
Discussion started by: Srini75
3 Replies

9. UNIX for Dummies Questions & Answers

excuting a shell script within ftp script

Novice here... I need help with excuting a shell script on a flat file that I've transfered over from a Windows XP machine for manipulation through an auto FTP script... so that after it is transfers it excutes the shell script and then returns it back to XP machine... Any ideas... (2 Replies)
Discussion started by: Gerry405
2 Replies

10. Shell Programming and Scripting

FTP script in Unix shell script

Hello , I am trying to make a shell script (Unix) for a ftp connection to another server and to get a file. So I have no knowledge about ftp and my script must do automaticly the connection with the user and passwd. Can you help us about that... Christian... (2 Replies)
Discussion started by: steiner
2 Replies
Login or Register to Ask a Question