bash script - sftpbatchfile - stop on failure


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting bash script - sftpbatchfile - stop on failure
# 8  
Old 04-04-2012
The manual pages refer to internal SFTP commands only. You are escaping to the shell on your local machine, so SFTP doesn't care what happens to it. This will try to delete the local file, which may be what you want, but you would be better closing the SFTP connection and then doing the remove of the local file in your shell script, where you can then test the return code.

Something like:-
Code:
$ cat sftp.cmnds
put localfile remotefile

$ cat myscript.sh
#!/bin/ksh
sftp a@b -b sftp.cmnds
RC=$?
if [ $RC -ne 0 ]
then
   ## Handle SFTP failure ##
else
   rm localfile
   RC=$?
   if [ $RC -ne 0 ]
   then
      ## Handle local file deletion error ##
   fi
fi


Does that help?


Robin
Liverpool/Blackburn
UK
# 9  
Old 04-04-2012
Quote:
Originally Posted by rbatte1
The manual pages refer to internal SFTP commands only. You are escaping to the shell on your local machine, so SFTP doesn't care what happens to it. This will try to delete the local file, which may be what you want, but you would be better closing the SFTP connection and then doing the remove of the local file in your shell script, where you can then test the return code.

Something like:-
Code:
$ cat sftp.cmnds
put localfile remotefile

$ cat myscript.sh
#!/bin/ksh
sftp a@b -b sftp.cmnds
RC=$?
if [ $RC -ne 0 ]
then
   ## Handle SFTP failure ##
else
   rm localfile
   RC=$?
   if [ $RC -ne 0 ]
   then
      ## Handle local file deletion error ##
   fi
fi

Does that help?


Robin
Liverpool/Blackburn
UK
This is what i first came up with, but there seems to be a problem when doing this.
Since it is a large amount of files the script makes a lot of different sftp connections. After an upgrade of the 3rth party server we are having many issues with sftp failures. Therefore i was charged to change the scripts so all files get sent over 1 sftp connection. (this way we avoid so many failures)
So bottom line, setting up a different connection for every file is out of the question.

(sorry i'm so difficult, but if it was easy i wouldn't have posted it ;-) ) Smilie
# 10  
Old 04-04-2012
As I mentioned in an edit above, can you do a pull rather than a push? Then you could use sftp's internal rm, and it would abort on fail.
# 11  
Old 04-04-2012
Ah. That rather shoots my suggestion out the water. I suppose you have already exhausted trying to find out why the SFTP doesn't always work.

You could have a retry loop to allow up to 5 failed attempts perhaps? Or, could you sleep for a second in between? I suppose that if there really are lots of files then that could add considerable time, so could you sleep for a second every 5 SFTPs? I suppose it depends on how frequently you get a failure.

Could it be a DNS/IP lookup issue at the remote server perhaps?


Okay, I'm probably waffling on about stuff you've already tried. Never mind.

I will keep thinking.



Robin
Liverpool/Blackburn
UK
# 12  
Old 04-04-2012
Quote:
Originally Posted by CarloM
As I mentioned in an edit above, can you do a pull rather than a push? Then you could use sftp's internal rm, and it would abort on fail.
that would indeed work, only as i mentioned before, the 3rd party server is a black box to us :-) (no possibility of getting scripts on it either Smilie)

---------- Post updated at 03:35 PM ---------- Previous update was at 03:33 PM ----------

Quote:
Originally Posted by rbatte1
Ah. That rather shoots my suggestion out the water. I suppose you have already exhausted trying to find out why the SFTP doesn't always work.

You could have a retry loop to allow up to 5 failed attempts perhaps? Or, could you sleep for a second in between? I suppose that if there really are lots of files then that could add considerable time, so could you sleep for a second every 5 SFTPs? I suppose it depends on how frequently you get a failure.

Could it be a DNS/IP lookup issue at the remote server perhaps?


Okay, I'm probably waffling on about stuff you've already tried. Never mind.

I will keep thinking.



Robin
Liverpool/Blackburn
UK
Indeed as you said, we have given up trying to figure out why it fails (not much help from 3rd party either). Their final answer was, we do not see any problems on our side. SmilieSmilieSmilie
# 13  
Old 04-04-2012
Quote:
Originally Posted by Kerberos
that would indeed work, only as i mentioned before, the 3rd party server is a black box to us :-) (no possibility of getting scripts on it either Smilie)
It would be so much nicer, however then you would need to change firewall settings to allow the other end to open a connection to your server and the potential risks there. I think you are right to dismiss SSH driving an SFTP back to you to do a get and rm as an option.



I see you have the usual "We're okay." type response. There must be a logging somewhere to say that the connection is refused, dropped, whatever. This is laziness on their side, especially if they made the change that broke it. I agree with your SmilieSmilieSmilie


No thoughts yet. Perhaps at 3am I will get an "A-ha!" moment.Smilie



Robin
Liverpool/Blackburn
UK
# 14  
Old 04-04-2012
Just to clarify, sftp -b isn't aborting on a failed put either?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

How to block first bash script until second bash script script launches web server/site?

I'm new to utilities like socat and netcat and I'm not clear if they will do what I need. I have a "compileDeployStartWebServer.sh" script and a "StartBrowser.sh" script that are started by emacs/elisp at the same time in two different processes. I'm using Cygwin bash on Windows 10. My... (3 Replies)
Discussion started by: siegfried
3 Replies

2. Shell Programming and Scripting

Bash variable assignment failure/unary operator expected

I have a little code block (executing on AIX 7.1) that I cannot understand why the NOTFREE=0 does not appear to be assigned even though it goes through that block. This causes a unary operator issue. #!/bin/bash PLATFORM="AIX" NEEDSPC=3000 set -x if ; then lsvg | grep -v rootvg | while... (6 Replies)
Discussion started by: port43
6 Replies

3. Debian

Bash script to STOP installation 'if' a file exists...

Hey all, Here's my dilemma: 1. I'm a newbie at scripting! 2. I need to create a script that checks: If a file size is equal to zero, then stop the installation. Is there a way to do this or am I wasting my time??? Thanx in advance! :b: (2 Replies)
Discussion started by: thazsar
2 Replies

4. UNIX for Dummies Questions & Answers

boot up failure unix sco after power failure

hi power went out. next day unix sco wont boot up error code 303. any help appreciated as we are clueless. (11 Replies)
Discussion started by: fredthayer
11 Replies

5. Shell Programming and Scripting

Stop child script by stoping parent script

Hi everyone, I have this problem with a script I'm writting. I want to execute a code running in the background several times through a script. I am writting it like that parent_script for a in 1 2 3 4 5 do exec test -n $a done What I want to do is when parent_script is killed,... (0 Replies)
Discussion started by: geovas
0 Replies

6. Shell Programming and Scripting

How to stop a script running in remote server from local script

Hi, I have googled for quite some time and couldn't able to get what exactly I am looking for.. My query is "how to stop a shell script which is running inside a remote server, using a script"??? can any one give some suggestions to sort this out. (1 Reply)
Discussion started by: mannepalli
1 Replies

7. UNIX for Dummies Questions & Answers

Anyone know?: How the 'for'-loop could stop working in interactive bash shell?!

It is happening with my sessions already second time: a 'for'-loop for some reason stop to work as expected. That means or it is looping without exitting, or it is not loop even once. Here example of my try when it is not processing even one loop. You can see, I start new subshell and... (14 Replies)
Discussion started by: alex_5161
14 Replies

8. Shell Programming and Scripting

script don't stop

Hello everybody! I am new to this and I am trying to change a script in an open source program that plots some offset vectors and then calls a postscript viewer. I have commented away the call for the postscript viewer but somehow the script doesn't return to the shell prompt. I cant figure out... (3 Replies)
Discussion started by: larne
3 Replies

9. UNIX for Dummies Questions & Answers

Stop a shell script

Hi, I am writing a bash shell script. How can I tell it to stop. For example, I would like to have something similar to the following: mike=1 if ; then STOP THE SCRIPT fi (3 Replies)
Discussion started by: msb65
3 Replies

10. Shell Programming and Scripting

Script does not stop when doing a read

Hi Folks, I have been trying to create a script wherein after it reads a certain number of data, it will pause and ask the user if he wants to continue or not. However, it seems that when it is supposed to read the user's answer, the script will go into a loop. What is wrong with my script here?... (7 Replies)
Discussion started by: rooseter
7 Replies
Login or Register to Ask a Question