Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Automated File Transfer Script Post 303008810 by Sidhant on Wednesday 6th of December 2017 03:01:10 PM
Old 12-06-2017
Automated File Transfer Script

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files

Script needs to go to each study name and then fetch the zipped files to another Windows location.

Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.

Or

we can fetch zip filenames and then insert into sql table and mark it as processed.

I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.

Code:
if [ $ftpmethod = SFTP_MGET_NOEXT ]
then
/usr/bin/expect -f - <<EOFMGET 
spawn sftp $remoteusername@$remotehost
expect "password:"
send "$remotepwd\n" 
expect "sftp> "
send "cd $remotedir\n"
expect "sftp> "
send "lcd $localdir\n"
expect "sftp> "
send "mget $filename*\n"
expect "sftp> "
send "sleep 120\n"
expect "sftp> "
send "exit\n"
interact
EOFMGET
fi

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 12-06-2017 at 04:08 PM.. Reason: Added CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX and Linux Applications

Automated file transfer

I want to automate the file transfer from a remote site to my pc over a WAN. Can anyone please suggest an industry standard secure file transfer tool for a windows environment? I am looking for an industry accepted, more secure than traditional FTP. Thanks! (1 Reply)
Discussion started by: damienjine
1 Replies

2. Shell Programming and Scripting

Perl automated file transfer

Hi, Firstly, I have no experience (at all) with shell scripting. So please, go easy on me :) I did a search for what I was looking for although there were a few things available here and on the net I couldn't find anything tailored to what I am looking for. Simply put, I have two servers.... (2 Replies)
Discussion started by: lastrider
2 Replies

3. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

4. Shell Programming and Scripting

Expect script file transfer failure

I use expect to spawn an sftp file transfer. On occasion the transfer fails or doesn't complete correctly, resulting in: "sftp> cd /returns sftp> rename /returns/TESTFILE.TXT /returns/archive/TESTFILE.TXT Couldn't rename file "/returns/TESTFILE.TXT" to "/returns/archive/TESTFILE.TXT": Failure... (0 Replies)
Discussion started by: tjb1959
0 Replies

5. Shell Programming and Scripting

Problem in Weekly file Transfer script

I have made a script which transfers some files of the entire week , but the script fails when the next month is started. For e.g; if i run the script on 5th may , but i need to transfer files of its previous week which is from 24th April to 30th april ,the script fails, i have this loop in the... (2 Replies)
Discussion started by: vee_789
2 Replies

6. Shell Programming and Scripting

Automated script to take 1000 records from the file every week.

I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday. I want script should should automatically take the next 1000 records for processing on next sunday. can we do it using head and tail head -1000 | tail... (2 Replies)
Discussion started by: sonam273
2 Replies

7. Shell Programming and Scripting

File transfer script

Hi, I need a shell script to transfer a file from one server(unix box) to another server(windows box). I have the details of the source and destination Ip's. source path : /home/UNIX/server filename:abc.txt Destination folder: D:/UNIX/test I am using AIX server. Type of shell :... (1 Reply)
Discussion started by: NareshN
1 Replies

8. Red Hat

UNIX script for daily file transfer ???

I need to create a script which needs to transfer the "<filename>+yyyymmdd".dat file from a unix machine to a linux machine.. daily. we can use an active batch to schedule the script. however, the script needs to detect if the new file is present, if present, then transfer. any help ?? (1 Reply)
Discussion started by: ravikodi
1 Replies

9. Shell Programming and Scripting

Deleting local server file from automated FTP script

Hi, I want to delete a file on the local server, while connected to remote server through FTP. I am using the below code for this $FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1 user $USER $PASSWORD cd $DIR ... (11 Replies)
Discussion started by: jhilmil
11 Replies

10. Shell Programming and Scripting

How to make script for file transfer?

Hi All, Please suggest me how to make script for file transfer from server X to another server Y. I have some directory path in server X as below: /home/directory_1/ . . /home/directory_n/ where some text files are available in each directory where some files records zero and some files... (6 Replies)
Discussion started by: aaditya321
6 Replies
libssh2_sftp_close_handle(3)					  libssh2 manual				      libssh2_sftp_close_handle(3)

NAME
libssh2_sftp_close_handle - close filehandle SYNOPSIS
#include <libssh2.h> #include <libssh2_sftp.h> int libssh2_sftp_close_handle(LIBSSH2_SFTP_HANDLE *handle); int libssh2_sftp_close(LIBSSH2_SFTP_HANDLE *handle); int libssh2_sftp_closedir(LIBSSH2_SFTP_HANDLE *handle); DESCRIPTION
handle - SFTP File Handle as returned by libssh2_sftp_open_ex(3) or libssh2_sftp_opendir(3) (which is a macro). Close an active LIBSSH2_SFTP_HANDLE. Because files and directories share the same underlying storage mechanism these methods may be used interchangably. libssh2_sftp_close(3) and libssh2_sftp_closedir(3) are macros for libssh2_sftp_close_handle(3). RETURN VALUE
Return 0 on success or negative on failure. It returns LIBSSH2_ERROR_EAGAIN when it would otherwise block. While LIBSSH2_ERROR_EAGAIN is a negative number, it isn't really a failure per se. ERRORS
LIBSSH2_ERROR_ALLOC - An internal memory allocation call failed. LIBSSH2_ERROR_SOCKET_SEND - Unable to send data on socket. LIBSSH2_ERROR_SOCKET_TIMEOUT - LIBSSH2_ERROR_SFTP_PROTOCOL - An invalid SFTP protocol response was received on the socket, or an SFTP operation caused an errorcode to be returned by the server. SEE ALSO
libssh2_sftp_open_ex(3) libssh2 0.15 1 Jun 2007 libssh2_sftp_close_handle(3)
All times are GMT -4. The time now is 10:51 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy