06-13-2017
Best way to transfer files to remote FTPS server instead of local FTPS server
Hi,
I am working on an application which runs on an Informatica Red-Hat 5.10 Linux Server.
The application involves several Informatica ETL workflows which generate 100s of Text files with lot of data. Many of the files will each be up to 5 GB in size.
Currently the Informatica server itself acts as an FTPS server and so these files are stored on the local file system by the ETL workflows and my application's downstream consumer applications use this Informatica server as their FTPS server and pull these data files from this server to their local client machines.
But going forward, due to space limitations and performance issues on Informatica server we are planning on acquiring a new Redhat Linux FTPS server and have these data files (which are gigabytes in size) generated by the ETL workflows on the local Informatica servers, pushed to the new remote FTPS server instead of storing them locally. Then I will have all downstream consumers connect to the new remote FTPS server to pull the data files they need. So then the informtica server will become a purely app server and no longer have the overhead of being an FTPS server
My question is what do you think is the best approach/strategy you would suggest for me to have the huge data files generated locally on my Informatica server shipped/transferred automatically to the new remote Linux FTPS server everytime they are generated. Since these files are huge, I know there will be latency in pushing the files from local Informatica server to remote FTPS server across network unlike before when they were being stored locally. Which Unix strategy or idea do you think would be best in minimizing these latency issues in shipping these huge files across the network to the remote FTPS server or not impact the SLAs' for file availability on the FTPS server too significantly so that downstream consumers applications do not see significant delays in file availability times. Lftp, rsync are few of the tools I have used. But are good enough for this purpose or is there some more creative approach that I could use. I know slight delays are inevitable since there is always a difference between storing files locally versus shipping them across the network to a remote FTPS server. But I am curious as to which approach will minimize this latency. Any inputs/ideas would be greatly appreciated.
thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies
2. HP-UX
want to remove the thread
thanks (2 Replies)
Discussion started by: indira
2 Replies
3. Shell Programming and Scripting
want to remove this thread.
thanks (2 Replies)
Discussion started by: indira
2 Replies
4. UNIX for Dummies Questions & Answers
Good evening
I need your help pease
I know there are 2 commands(diff, or cp) to compare files in a directory. but the question arises:
1. can i compare zip files or ive got to unzip them?
2. how can i compare 2 files from a local to a remote server? (is there any special commad or ive got... (4 Replies)
Discussion started by: alexcol
4 Replies
5. UNIX for Dummies Questions & Answers
Hi experts,
I 'm newbie to unix world, now I have task to copy the latest files from remote server to my local. I believe this must be very common request in this community. I want you do it one more time for me please.
My requirement is something like this:
I receive files in the below... (3 Replies)
Discussion started by: parpaa
3 Replies
6. Solaris
Hi,
I am trying to automate the process of fetching files from remote server to local server through sftp. I have the username and password for the remote solaris server. But I need to give password manually everytime i run the script.
Can anyone help me in automating the script such that it... (3 Replies)
Discussion started by: ssk250
3 Replies
7. UNIX for Dummies Questions & Answers
Hi All,
I am new to SHell scripting, can someone please help me with the below requirement.
1) LFTP a file to a remote server which supports FTPS. My current enviroment is Sun Solaris 5.10
2) I need to incorporate this in a shell which is currently sending files to a server that accepts... (1 Reply)
Discussion started by: subbu
1 Replies
8. Shell Programming and Scripting
Hi All
I need to transfer a file from a UNIX server to a windows server.
I saw that it is possible to do this using scp command by looking at the forum listed below:
... (2 Replies)
Discussion started by: vx04
2 Replies
9. Shell Programming and Scripting
Hi Guyz ,,
I'm an ERP functional guy , I really need to automate a script which required a shell script but have a little knowledge in shell scripting.
I need my generated files to be zipped first in one directory lets say (xyz) and then it needs to transfer another ftp server in... (3 Replies)
Discussion started by: Shogundion
3 Replies
10. Shell Programming and Scripting
Hi Team,
I am currently working on an integration project where in we have planned to receive files from an external source onto our system via FTPS(SSL). I am new to this part and would like to know the points for consideration and the points to learn to get this done. The files we receive from... (1 Reply)
Discussion started by: Rads
1 Replies
LEARN ABOUT MOJAVE
curlopt_sslcerttype
CURLOPT_SSLCERTTYPE(3) curl_easy_setopt options CURLOPT_SSLCERTTYPE(3)
NAME
CURLOPT_SSLCERTTYPE - specify type of the client SSL certificate
SYNOPSIS
#include <curl/curl.h>
CURLcode curl_easy_setopt(CURL *handle, CURLOPT_SSLCERTTYPE, char *type);
DESCRIPTION
Pass a pointer to a zero terminated string as parameter. The string should be the format of your certificate. Supported formats are "PEM"
and "DER", except with Secure Transport. OpenSSL (versions 0.9.3 and later) and Secure Transport (on iOS 5 or later, or OS X 10.7 or later)
also support "P12" for PKCS#12-encoded files.
The application does not have to keep the string around after setting this option.
DEFAULT
"PEM"
PROTOCOLS
All TLS based protocols: HTTPS, FTPS, IMAPS, POP3S, SMTPS etc.
EXAMPLE
TODO
AVAILABILITY
If built TLS enabled. Added in 7.9.3
RETURN VALUE
Returns CURLE_OK if TLS is supported, CURLE_UNKNOWN_OPTION if not, or CURLE_OUT_OF_MEMORY if there was insufficient heap space.
SEE ALSO
CURLOPT_SSLCERT(3), CURLOPT_SSLKEY(3),
libcurl 7.54.0 December 21, 2016 CURLOPT_SSLCERTTYPE(3)