Improve sftp script


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Improve sftp script
# 1  
Old 11-06-2014
Improve sftp script

Dear all,

I have written two scripts to transfer files to another server outside the company.
One is a batch script , and the other script calls the batch script, send the files and archive the file sent.

The problem is, that I want to get the list of files which have been uploaded the the other server, in order to be sure that all files have been transferred.

Below is the batch script, and the other script.

Please, would you take a look and give any suggestion to improve the performance of these scripts?

Thank you in advance!!

batch-script:

Code:
UploadFile=$1
ServerSftp=$2
echo put $UploadFile > /tmp/batch-sftp$$.log
# Send the file throught SFTP as Batch
sftp -b /tmp/batch-sftp$$.log $ServerSftp
# Remove log file
rm -f /tmp/batch-sftp$$.log

push script:

Code:
$data = `date +%c`;
chomp($data);

# get the list of files.
@files = </usr/local/xxx/yyy/zzz/*>;

foreach $file (@files) {
# Send each file with SFTP
$sftp_put = `/usr/local/scripts/batch-sftp.sh "$file" username\@192.168.xx:xx/upload 1> /dev/null; echo \$?`;
# clean out-put from newline
chomp($sftp_put);

        if ($sftp_put eq "0") {
                $move_archive = `mv "$file" /usr/local/archive/ > /dev/null; echo \$?`;
                    chomp($leviz_archive);
                                              if ($move_archive == "0") {
                                $write_logs = `echo \"$data - OK - "$file" was sucessfully uploaded\" >> /usr/local/logs/outgoing.log`;

                                $write_logs = `echo \"$data - INFO - "$file" was sucessfully moved to /usr/local/archive/\" >> /usr/local/logs/outgoing.log`;
                           } else {
                                 $write_logs = `echo \"$data - ERROR - "$file" cannot be moved to /usr/local/archive/ : "$move_archive"\"  >> /usr/local/logs/outgoing.log`;
                        }
        } elsif ($sftp_put ne "0") {
             $write_logs = `echo \"$data - ERROR - "$file" cannot be uploaded : "$sftp_put"\" >> /usr/local/logs/outgoing.log`;
        }
 }


Last edited by rbatte1; 11-06-2014 at 08:25 AM.. Reason: Capital letters
# 2  
Old 11-06-2014
Why not use sftp's capabilities like "abort on error" or "raised logging level" (man sftp)
Quote:
-b batchfile
Batch mode reads a series of commands from an input batchfile instead of stdin. Since it lacks user interaction it should be
used in conjunction with non-interactive authentication. A batchfile of ‘-’ may be used to indicate standard input. sftp
will abort if any of the following commands fail: get, put, reget, rename, ln, rm, mkdir, chdir, ls, lchdir, chmod, chown,
chgrp, lpwd, df, symlink, and lmkdir. Termination on error can be suppressed on a command by command basis by prefixing the
command with a ‘-’ character (for example, -rm /tmp/blah*).

-v Raise logging level. This option is also passed to ssh.
to make sure all files have been transmitted correctly instead of that cumbersome later analysis? Create the batchfile carefully, and then check the logfile for anomalies.
# 3  
Old 11-06-2014
Thank you for replying!
Please, would you help me on editing this script?
# 4  
Old 11-06-2014
Is there a particular reason you have a perl script program and a shell script? Could this all be wrapped up into 1 shell script?
# 5  
Old 11-07-2014
Quote:
Originally Posted by arrals_vl
Thank you for replying!
Please, would you help me on editing this script?
That's not too easy remotely. Create a test batch file, use
Code:
sftp -vb batchfile >logfile 2>&1
echo $?

, do the same with some errors in the batch file (e.g. non-existing files) and then compare/post the logfiles.
# 6  
Old 11-07-2014
Dear Rudic, thank you, i will try it.

@Chubler_XL Chubler_XL Thank you. Not, i can wrapped it into 1 shell script.
# 7  
Old 11-12-2014
Dear all,

Please any help on this?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Improve script

Gents, Is there the possibility to improve this script to be able to have same output information. I did this script, but I believe there is a very short code to get same output here my script awk -F, '{if($10>0 && $10<=15) print $6}' tmp1 | sort -k1n | awk '{a++} END { for (n in a )... (23 Replies)
Discussion started by: jiam912
23 Replies

2. Shell Programming and Scripting

Improve script and get new output file

Gents, Using the following script, I got the changes as desired in the output file called (spread_2611.x01.new). Complete file as input (spread_2611.x01). Can you please have a look to my script and improve it please. :b: Also I would like to I get a additional selecting only the records... (21 Replies)
Discussion started by: jiam912
21 Replies

3. Shell Programming and Scripting

How to improve an script?

Gents. I have 2 different scripts for the same purpose: raw2csv_1 Script raw2csv_1 finish the process in less that 1 minute raw2csv_2 Script raw2csv_2 finish the process in more that 6 minutes. Can you please check if there is any option to improve the raw2csv_2. To finish the job... (4 Replies)
Discussion started by: jiam912
4 Replies

4. UNIX for Dummies Questions & Answers

How to improve the performance of this script?

Hi , i wrote a script to convert dates to the formate i want .it works fine but the conversion is tkaing lot of time . Can some one help me tweek this script #!/bin/bash file=$1 ofile=$2 cp $file $ofile mydates=$(grep -Po '+/+/+' $ofile) # gets 8/1/13 mydates=$(echo "$mydates" | sort |... (5 Replies)
Discussion started by: vikatakavi
5 Replies

5. Shell Programming and Scripting

Var Check Script (Help improve if possible)

I am working on a script to check the var on all of my systems. Can someone help me fix it to work better or give me suggestions. #!/bin/ksh IN=/path/to/list_of_workstations.txt while read hostnames do if ping $hostnames 1 | grep alive > /dev/null then percent=`ssh -q... (3 Replies)
Discussion started by: whotippedmycow
3 Replies

6. UNIX for Dummies Questions & Answers

[please] improve my shell/SQL*Plus script

Hi We generate with PL/SQL *.csv files, archive them and mail to the customer. Here is my script (Solaris 10, ksh): #!/bin/ksh # Unix Shell Script Structure for PL/SQL queries with SQL*Plus . ~/.profile scriptdir=/opt/ora/scripts queryname1=example... (1 Reply)
Discussion started by: slashdotweenie
1 Replies

7. Shell Programming and Scripting

Want to improve the performance of script

Hi All, I have written a script as follows which is taking lot of time in executing/searching only 3500 records taken as input from one file in log file of 12 GB Approximately. Working of script is read the csv file as an input having 2 arguments which are transaction_id,mobile_number and search... (6 Replies)
Discussion started by: poweroflinux
6 Replies

8. Shell Programming and Scripting

Improve the performance of a shell script

Hi Friends, I wrote the below shell script to generate a report on alert messages recieved on a day. But i for processing around 4500 lines (alerts) the script is taking aorund 30 minutes to process. Please help me to make it faster and improve the performace of the script. i would be very... (10 Replies)
Discussion started by: apsprabhu
10 Replies

9. Shell Programming and Scripting

Any way to improve performance of this script

I have a data file of 2 gig I need to do all these, but its taking hours, any where i can improve performance, thanks a lot #!/usr/bin/ksh echo TIMESTAMP="$(date +'_%y-%m-%d.%H-%M-%S')" function showHelp { cat << EOF >&2 syntax extreme.sh FILENAME Specify filename to parse EOF... (3 Replies)
Discussion started by: sirababu
3 Replies

10. Shell Programming and Scripting

Can I improve this script ???

Hi all, Still a newbie and learning as I go ... as you do :) Have created this script to report on disc usage and I've just included the ChkSpace function this morning. It's the first time I've read a file (line-by-bloody-line) and would like to know if I can improve this script ? FYI - I... (11 Replies)
Discussion started by: Cameron
11 Replies
Login or Register to Ask a Question