Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Automated File Transfer Script Post 303008810 by Sidhant on Wednesday 6th of December 2017 03:01:10 PM
Old 12-06-2017
Automated File Transfer Script

We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files

Script needs to go to each study name and then fetch the zipped files to another Windows location.

Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.

Or

we can fetch zip filenames and then insert into sql table and mark it as processed.

I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.

Code:
if [ $ftpmethod = SFTP_MGET_NOEXT ]
then
/usr/bin/expect -f - <<EOFMGET 
spawn sftp $remoteusername@$remotehost
expect "password:"
send "$remotepwd\n" 
expect "sftp> "
send "cd $remotedir\n"
expect "sftp> "
send "lcd $localdir\n"
expect "sftp> "
send "mget $filename*\n"
expect "sftp> "
send "sleep 120\n"
expect "sftp> "
send "exit\n"
interact
EOFMGET
fi

Moderator's Comments:
Mod Comment Please use CODE tags as required by forum rules!

Last edited by RudiC; 12-06-2017 at 04:08 PM.. Reason: Added CODE tags.
 

10 More Discussions You Might Find Interesting

1. UNIX and Linux Applications

Automated file transfer

I want to automate the file transfer from a remote site to my pc over a WAN. Can anyone please suggest an industry standard secure file transfer tool for a windows environment? I am looking for an industry accepted, more secure than traditional FTP. Thanks! (1 Reply)
Discussion started by: damienjine
1 Replies

2. Shell Programming and Scripting

Perl automated file transfer

Hi, Firstly, I have no experience (at all) with shell scripting. So please, go easy on me :) I did a search for what I was looking for although there were a few things available here and on the net I couldn't find anything tailored to what I am looking for. Simply put, I have two servers.... (2 Replies)
Discussion started by: lastrider
2 Replies

3. UNIX for Advanced & Expert Users

Automated SCP script passing password to preserve source file timestamp

Hi My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file. By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
Discussion started by: skumar75
1 Replies

4. Shell Programming and Scripting

Expect script file transfer failure

I use expect to spawn an sftp file transfer. On occasion the transfer fails or doesn't complete correctly, resulting in: "sftp> cd /returns sftp> rename /returns/TESTFILE.TXT /returns/archive/TESTFILE.TXT Couldn't rename file "/returns/TESTFILE.TXT" to "/returns/archive/TESTFILE.TXT": Failure... (0 Replies)
Discussion started by: tjb1959
0 Replies

5. Shell Programming and Scripting

Problem in Weekly file Transfer script

I have made a script which transfers some files of the entire week , but the script fails when the next month is started. For e.g; if i run the script on 5th may , but i need to transfer files of its previous week which is from 24th April to 30th april ,the script fails, i have this loop in the... (2 Replies)
Discussion started by: vee_789
2 Replies

6. Shell Programming and Scripting

Automated script to take 1000 records from the file every week.

I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday. I want script should should automatically take the next 1000 records for processing on next sunday. can we do it using head and tail head -1000 | tail... (2 Replies)
Discussion started by: sonam273
2 Replies

7. Shell Programming and Scripting

File transfer script

Hi, I need a shell script to transfer a file from one server(unix box) to another server(windows box). I have the details of the source and destination Ip's. source path : /home/UNIX/server filename:abc.txt Destination folder: D:/UNIX/test I am using AIX server. Type of shell :... (1 Reply)
Discussion started by: NareshN
1 Replies

8. Red Hat

UNIX script for daily file transfer ???

I need to create a script which needs to transfer the "<filename>+yyyymmdd".dat file from a unix machine to a linux machine.. daily. we can use an active batch to schedule the script. however, the script needs to detect if the new file is present, if present, then transfer. any help ?? (1 Reply)
Discussion started by: ravikodi
1 Replies

9. Shell Programming and Scripting

Deleting local server file from automated FTP script

Hi, I want to delete a file on the local server, while connected to remote server through FTP. I am using the below code for this $FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1 user $USER $PASSWORD cd $DIR ... (11 Replies)
Discussion started by: jhilmil
11 Replies

10. Shell Programming and Scripting

How to make script for file transfer?

Hi All, Please suggest me how to make script for file transfer from server X to another server Y. I have some directory path in server X as below: /home/directory_1/ . . /home/directory_n/ where some text files are available in each directory where some files records zero and some files... (6 Replies)
Discussion started by: aaditya321
6 Replies
Net::SFTP::Foreign::Compat(3pm) 			User Contributed Perl Documentation			   Net::SFTP::Foreign::Compat(3pm)

NAME
Net::SFTP::Foreign::Compat - Adaptor for Net::SFTP compatibility SYNOPSIS
use Net::SFTP::Foreign::Compat; my $sftp = Net::SFTP::Foreign::Compat->new($host); $sftp->get("foo", "bar"); $sftp->put("bar", "baz"); use Net::SFTP::Foreign::Compat ':supplant'; my $sftp = Net::SFTP->new($host); DESCRIPTION
This package is a wrapper around Net::SFTP::Foreign that provides an API (mostly) compatible with that of Net::SFTP. Methods on this package are identical to those in Net::SFTP except that Net::SFTP::Foreign::Attributes::Compat objects have to be used instead of Net::SFTP::Attributes. If the ":supplant" tag is used, this module installs also wrappers on the "Net::SFTP" and Net::SFTP::Attributes packages so no other parts of the program have to modified in order to move from Net::SFTP to Net::SFTP::Foreign. Setting defaults The hash %Net::SFTP::Foreign::DEFAULTS can be used to set default values for Net::SFTP::Foreign methods called under the hood and otherwise not accesible through the Net::SFTP API. The entries currently supported are: new => @opts extra options passed to Net::SFTP::Foreign constructor. get => @opts extra options passed to Net::SFTP::Foreign::get method. put => @opts extra options passed to Net::SFTP::Foreign::put method. ls => @opts extra options passed to Net::SFTP::Foreign::ls method. COPYRIGHT
Copyright (c) 2006-2008, 2011 Salvador Fandin~o All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.14.2 2012-05-04 Net::SFTP::Foreign::Compat(3pm)
All times are GMT -4. The time now is 11:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy