Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Automated File Transfer Script Post 303008810 by Sidhant on Wednesday 6th of December 2017 03:01:10 PM
Old 12-06-2017
Automated File Transfer Script

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files

Script needs to go to each study name and then fetch the zipped files to another Windows location.

Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.

Or

we can fetch zip filenames and then insert into sql table and mark it as processed.

I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.

Code:
if [ $ftpmethod = SFTP_MGET_NOEXT ]
then
/usr/bin/expect -f - <<EOFMGET 
spawn sftp $remoteusername@$remotehost
expect "password:"
send "$remotepwd\n" 
expect "sftp> "
send "cd $remotedir\n"
expect "sftp> "
send "lcd $localdir\n"
expect "sftp> "
send "mget $filename*\n"
expect "sftp> "
send "sleep 120\n"
expect "sftp> "
send "exit\n"
interact
EOFMGET
fi

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 12-06-2017 at 04:08 PM.. Reason: Added CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX and Linux Applications

Automated file transfer

I want to automate the file transfer from a remote site to my pc over a WAN. Can anyone please suggest an industry standard secure file transfer tool for a windows environment? I am looking for an industry accepted, more secure than traditional FTP. Thanks! (1 Reply)
Discussion started by: damienjine
1 Replies

2. Shell Programming and Scripting

Perl automated file transfer

Hi, Firstly, I have no experience (at all) with shell scripting. So please, go easy on me :) I did a search for what I was looking for although there were a few things available here and on the net I couldn't find anything tailored to what I am looking for. Simply put, I have two servers.... (2 Replies)
Discussion started by: lastrider
2 Replies

3. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

4. Shell Programming and Scripting

Expect script file transfer failure

I use expect to spawn an sftp file transfer. On occasion the transfer fails or doesn't complete correctly, resulting in: "sftp> cd /returns sftp> rename /returns/TESTFILE.TXT /returns/archive/TESTFILE.TXT Couldn't rename file "/returns/TESTFILE.TXT" to "/returns/archive/TESTFILE.TXT": Failure... (0 Replies)
Discussion started by: tjb1959
0 Replies

5. Shell Programming and Scripting

Problem in Weekly file Transfer script

I have made a script which transfers some files of the entire week , but the script fails when the next month is started. For e.g; if i run the script on 5th may , but i need to transfer files of its previous week which is from 24th April to 30th april ,the script fails, i have this loop in the... (2 Replies)
Discussion started by: vee_789
2 Replies

6. Shell Programming and Scripting

Automated script to take 1000 records from the file every week.

I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday. I want script should should automatically take the next 1000 records for processing on next sunday. can we do it using head and tail head -1000 | tail... (2 Replies)
Discussion started by: sonam273
2 Replies

7. Shell Programming and Scripting

File transfer script

Hi, I need a shell script to transfer a file from one server(unix box) to another server(windows box). I have the details of the source and destination Ip's. source path : /home/UNIX/server filename:abc.txt Destination folder: D:/UNIX/test I am using AIX server. Type of shell :... (1 Reply)
Discussion started by: NareshN
1 Replies

8. Red Hat

UNIX script for daily file transfer ???

I need to create a script which needs to transfer the "<filename>+yyyymmdd".dat file from a unix machine to a linux machine.. daily. we can use an active batch to schedule the script. however, the script needs to detect if the new file is present, if present, then transfer. any help ?? (1 Reply)
Discussion started by: ravikodi
1 Replies

9. Shell Programming and Scripting

Deleting local server file from automated FTP script

Hi, I want to delete a file on the local server, while connected to remote server through FTP. I am using the below code for this $FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1 user $USER $PASSWORD cd $DIR ... (11 Replies)
Discussion started by: jhilmil
11 Replies

10. Shell Programming and Scripting

How to make script for file transfer?

Hi All, Please suggest me how to make script for file transfer from server X to another server Y. I have some directory path in server X as below: /home/directory_1/ . . /home/directory_n/ where some text files are available in each directory where some files records zero and some files... (6 Replies)
Discussion started by: aaditya321
6 Replies
Net::SFTP::Foreign::Constants(3pm)			User Contributed Perl Documentation			Net::SFTP::Foreign::Constants(3pm)

NAME
Net::SFTP::Foreign::Constants - Constant definitions for Net::SFTP::Foreign SYNOPSIS
use Net::SFTP::Foreign::Constants qw(:tag SSH2_FILEXFER_VERSION); print "Protocol version is ", SSH2_FILEXFER_VERSION; DESCRIPTION
Net::SFTP::Foreign::Constants provides a list of exportable SFTP constants: for SFTP messages and commands, for file-open flags, for status messages, etc. Constants can be exported individually, or in sets identified by tag names. Net::SFTP::Foreign::Constants provides values for all of the constants listed in the SFTP protocol version 3 draft; the only thing to note is that the constants are listed with the prefix "SSH2_" instead of "SSH_". So, for example, to import the constant for the file-open command, you would write: use Net::SFTP::Foreign::Constants qw( SSH2_FXP_OPEN ); TAGS
As mentioned above, constants can either be imported individually or in sets grouped by tag names. The tag names are: :fxp Imports all of the "SSH2_FXP_*" constants: these are the constants used in the messaging protocol. :flags Imports all of the "SSH2_FXF_*" constants: these are constants used as flags sent to the server when opening files. :att Imports all of the "SSH2_FILEXFER_ATTR_*" constants: these are the constants used to construct the flag in the serialized attributes. The flag describes what types of file attributes are listed in the buffer. :status Imports all of the "SSH2_FX_*" constants: these are constants returned from a server "SSH2_FXP_STATUS" message and indicate the status of a particular operation. :error Imports all the "SFTP_ERR_*" constants used to represent high level errors: "SFTP_ERR_LOCAL_ALREADY_EXISTS", "SFTP_ERR_LOCAL_CHMOD_FAILED", "SFTP_ERR_LOCAL_OPEN_FAILED", "SFTP_ERR_LOCAL_READ_ERROR", "SFTP_ERR_LOCAL_STAT_FAILED", "SFTP_ERR_LOCAL_UTIME_FAILED", "SFTP_ERR_LOCAL_WRITE_FAILED", "SFTP_ERR_REMOTE_BAD_PACKET_SEQUENCE", "SFTP_ERR_REMOTE_BAD_PERMISSIONS", "SFTP_ERR_REMOTE_BAD_TIME", "SFTP_ERR_REMOTE_BLOCK_TOO_SMALL", "SFTP_ERR_REMOTE_CLOSE_FAILED", "SFTP_ERR_REMOTE_FSETSTAT_FAILED", "SFTP_ERR_REMOTE_FSTAT_FAILED", "SFTP_ERR_REMOTE_LSTAT_FAILED", "SFTP_ERR_REMOTE_MKDIR_FAILED", "SFTP_ERR_REMOTE_OPENDIR_FAILED", "SFTP_ERR_REMOTE_OPEN_FAILED", "SFTP_ERR_REMOTE_READDIR_FAILED", "SFTP_ERR_REMOTE_READ_FAILED", "SFTP_ERR_REMOTE_REALPATH_FAILED", "SFTP_ERR_REMOTE_REMOVE_FAILED", "SFTP_ERR_REMOTE_RENAME_FAILED", "SFTP_ERR_REMOTE_RMDIR_FAILED", "SFTP_ERR_REMOTE_READLINK_FAILED", "SFTP_ERR_REMOTE_SYMLINK_FAILED", "SFTP_ERR_REMOTE_SETSTAT_FAILED", "SFTP_ERR_REMOTE_STAT_FAILED", "SFTP_ERR_REMOTE_WRITE_FAILED" and "SFTP_ERR_REMOTE_HARDLINK_FAILED". Note: these constants are not defined on the SFTP draft. :ext Import all the "SSH2_FXE_*" constants: there are the constants defined for usage with SFTP extensions. Currently, these are supported: "SSH2_FXE_STATVFS_ST_RDONLY", "SSH_FXE_STATVFS_ST_NOSUID". There is one constant that does not fit into any of the tag sets: "SSH2_FILEXFER_VERSION", which holds the value of the SFTP protocol implemented by Net::SFTP::Foreign. AUTHOR &; COPYRIGHTS Please see the Net::SFTP::Foreign manpage for author, copyright, and license information. perl v5.14.2 2012-05-11 Net::SFTP::Foreign::Constants(3pm)
All times are GMT -4. The time now is 12:33 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy