We are receiving data feed files in SFTP location daily. so the folder structure in SFTP location is abc/def/studyname_1/outbound/zipped files
So we will be getting different studies and for each study a folder is created abc/def/studyname_2/outbound/zipped files , abc/def/studyname_3/outbound/zipped files
Script needs to go to each study name and then fetch the zipped files to another Windows location.
Here I would want to have a reference table created in SQL where I have listed all the incoming zipped files and its SFTP location. So the script also needs to cross verify with the sql table and then pick-up the files.
Or
we can fetch zip filenames and then insert into sql table and mark it as processed.
I understand we can use MGET to fetch all the files in SFTP location. I am confused on how I can achieve the tracking of the zip files and how to go through each study and fetch files.
Moderator's Comments:
Please use CODE tags as required by forum rules!
Last edited by RudiC; 12-06-2017 at 04:08 PM..
Reason: Added CODE tags.
I want to automate the file transfer from a remote site to my pc over a WAN. Can anyone please suggest an industry standard secure file transfer tool for a windows environment?
I am looking for an industry accepted, more secure than traditional FTP.
Thanks! (1 Reply)
Hi,
Firstly, I have no experience (at all) with shell scripting. So please, go easy on me :)
I did a search for what I was looking for although there were a few things available here and on the net I couldn't find anything tailored to what I am looking for.
Simply put, I have two servers.... (2 Replies)
Hi
My requirement is i want to copy files from remote server to the local server and also i need to preserve the timestamp of the remote file.
By using scp -p , it is working fine in the interactive call but it is not preserving he file timestamp when i use it in the non interactive scp call... (1 Reply)
I use expect to spawn an sftp file transfer. On occasion the transfer fails or doesn't complete correctly, resulting in:
"sftp> cd /returns
sftp> rename /returns/TESTFILE.TXT /returns/archive/TESTFILE.TXT
Couldn't rename file "/returns/TESTFILE.TXT" to "/returns/archive/TESTFILE.TXT":
Failure... (0 Replies)
I have made a script which transfers some files of the entire week , but the script fails when the next month is started.
For e.g; if i run the script on 5th may , but i need to transfer files of its previous week which is from 24th April to 30th april ,the script fails, i have this loop in the... (2 Replies)
I have a file having n number of records .i want first 1000 records from the file and store in temporary file to process on sunday.
I want script should should automatically take the next 1000 records for processing on next sunday.
can we do it using head and tail
head -1000 | tail... (2 Replies)
Hi,
I need a shell script to transfer a file from one server(unix box) to another server(windows box).
I have the details of the source and destination Ip's.
source path : /home/UNIX/server
filename:abc.txt
Destination folder: D:/UNIX/test
I am using AIX server.
Type of shell :... (1 Reply)
I need to create a script which needs to transfer the "<filename>+yyyymmdd".dat file from a unix machine to a linux machine.. daily. we can use an active batch to schedule the script. however, the script needs to detect if the new file is present, if present, then transfer.
any help ?? (1 Reply)
Hi,
I want to delete a file on the local server, while connected to remote server through FTP.
I am using the below code for this
$FTP_CMD -v -n $HOST <<*! >> $LOGFILE 2>&1
user $USER $PASSWORD
cd $DIR
... (11 Replies)
Hi All,
Please suggest me how to make script for file transfer from server X to another server Y.
I have some directory path in server X as below:
/home/directory_1/
.
.
/home/directory_n/
where some text files are available in each directory where some files records zero and some files... (6 Replies)
Discussion started by: aaditya321
6 Replies
LEARN ABOUT MOJAVE
curlopt_ftp_create_missing_dirs
CURLOPT_FTP_CREATE_MISSING_DIRS(3) curl_easy_setopt options CURLOPT_FTP_CREATE_MISSING_DIRS(3)NAME
CURLOPT_FTP_CREATE_MISSING_DIRS - create missing dirs for FTP and SFTP
SYNOPSIS
#include <curl/curl.h>
typedef enum {
CURLFTP_CREATE_DIR_NONE,
CURLFTP_CREATE_DIR,
CURLFTP_CREATE_DIR_RETRY
} curl_ftpcreatedir;
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_FTP_CREATE_MISSING_DIRS,
long create);
DESCRIPTION
Pass a long telling libcurl to create the dir. If the value is CURLFTP_CREATE_DIR (1), libcurl will attempt to create any remote directory
that it fails to "move" into.
For FTP requests, that means a CWD command fails. CWD being the command that changes working directory.
For SFTP requests, libcurl will attempt to create the remote directory if it can't obtain a handle to the target-location. The creation
will fail if a file of the same name as the directory to create already exists or lack of permissions prevents creation.
Setting create to CURLFTP_CREATE_DIR_RETRY (2), tells libcurl to retry the CWD command again if the subsequent MKD command fails. This is
especially useful if you're doing many simultaneous connections against the same server and they all have this option enabled, as then CWD
may first fail but then another connection does MKD before this connection and thus MKD fails but trying CWD works!
DEFAULT
CURLFTP_CREATE_DIR_NONE (0)
PROTOCOLS
FTP and SFTP
EXAMPLE
TODO
AVAILABILITY
Added in 7.10.7. SFTP support added in 7.16.3. The retry option was added in 7.19.4.
RETURN VALUE
Returns CURLE_OK if the option is supported, and CURLE_UNKNOWN_OPTION if the create value is not.
SEE ALSO CURLOPT_FTP_FILEMETHOD(3), CURLOPT_FTP_USE_EPSV(3),
libcurl 7.54.0 February 03, 2016 CURLOPT_FTP_CREATE_MISSING_DIRS(3)