06-13-2017
Best way to transfer files to remote FTPS server instead of local FTPS server
Hi,
I am working on an application which runs on an Informatica Red-Hat 5.10 Linux Server.
The application involves several Informatica ETL workflows which generate 100s of Text files with lot of data. Many of the files will each be up to 5 GB in size.
Currently the Informatica server itself acts as an FTPS server and so these files are stored on the local file system by the ETL workflows and my application's downstream consumer applications use this Informatica server as their FTPS server and pull these data files from this server to their local client machines.
But going forward, due to space limitations and performance issues on Informatica server we are planning on acquiring a new Redhat Linux FTPS server and have these data files (which are gigabytes in size) generated by the ETL workflows on the local Informatica servers, pushed to the new remote FTPS server instead of storing them locally. Then I will have all downstream consumers connect to the new remote FTPS server to pull the data files they need. So then the informtica server will become a purely app server and no longer have the overhead of being an FTPS server
My question is what do you think is the best approach/strategy you would suggest for me to have the huge data files generated locally on my Informatica server shipped/transferred automatically to the new remote Linux FTPS server everytime they are generated. Since these files are huge, I know there will be latency in pushing the files from local Informatica server to remote FTPS server across network unlike before when they were being stored locally. Which Unix strategy or idea do you think would be best in minimizing these latency issues in shipping these huge files across the network to the remote FTPS server or not impact the SLAs' for file availability on the FTPS server too significantly so that downstream consumers applications do not see significant delays in file availability times. Lftp, rsync are few of the tools I have used. But are good enough for this purpose or is there some more creative approach that I could use. I know slight delays are inevitable since there is always a difference between storing files locally versus shipping them across the network to a remote FTPS server. But I am curious as to which approach will minimize this latency. Any inputs/ideas would be greatly appreciated.
thanks
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies
2. HP-UX
want to remove the thread
thanks (2 Replies)
Discussion started by: indira
2 Replies
3. Shell Programming and Scripting
want to remove this thread.
thanks (2 Replies)
Discussion started by: indira
2 Replies
4. UNIX for Dummies Questions & Answers
Good evening
I need your help pease
I know there are 2 commands(diff, or cp) to compare files in a directory. but the question arises:
1. can i compare zip files or ive got to unzip them?
2. how can i compare 2 files from a local to a remote server? (is there any special commad or ive got... (4 Replies)
Discussion started by: alexcol
4 Replies
5. UNIX for Dummies Questions & Answers
Hi experts,
I 'm newbie to unix world, now I have task to copy the latest files from remote server to my local. I believe this must be very common request in this community. I want you do it one more time for me please.
My requirement is something like this:
I receive files in the below... (3 Replies)
Discussion started by: parpaa
3 Replies
6. Solaris
Hi,
I am trying to automate the process of fetching files from remote server to local server through sftp. I have the username and password for the remote solaris server. But I need to give password manually everytime i run the script.
Can anyone help me in automating the script such that it... (3 Replies)
Discussion started by: ssk250
3 Replies
7. UNIX for Dummies Questions & Answers
Hi All,
I am new to SHell scripting, can someone please help me with the below requirement.
1) LFTP a file to a remote server which supports FTPS. My current enviroment is Sun Solaris 5.10
2) I need to incorporate this in a shell which is currently sending files to a server that accepts... (1 Reply)
Discussion started by: subbu
1 Replies
8. Shell Programming and Scripting
Hi All
I need to transfer a file from a UNIX server to a windows server.
I saw that it is possible to do this using scp command by looking at the forum listed below:
... (2 Replies)
Discussion started by: vx04
2 Replies
9. Shell Programming and Scripting
Hi Guyz ,,
I'm an ERP functional guy , I really need to automate a script which required a shell script but have a little knowledge in shell scripting.
I need my generated files to be zipped first in one directory lets say (xyz) and then it needs to transfer another ftp server in... (3 Replies)
Discussion started by: Shogundion
3 Replies
10. Shell Programming and Scripting
Hi Team,
I am currently working on an integration project where in we have planned to receive files from an external source onto our system via FTPS(SSL). I am new to this part and would like to know the points for consideration and the points to learn to get this done. The files we receive from... (1 Reply)
Discussion started by: Rads
1 Replies
LEARN ABOUT CENTOS
radtest
RADTEST(1) FreeRADIUS Daemon RADTEST(1)
NAME
radtest - send packets to a RADIUS server, show reply
SYNOPSIS
radtest [-d raddb_directory] [-t pap/chap/mschap] [-x ] [-4 ] [-6 ] user password radius-server nas-port-number secret [ppphint] [nasname]
DESCRIPTION
radtest is a frontend to radclient(1). It generates a list of attribute/value pairs based on the command line arguments, and feeds these
into radclient. It's a fast and convenient way to test a radius server.
OPTIONS
-d raddb_directory
The directory that contains the RADIUS dictionary files. Defaults to /etc/raddb.
-t pap/chap/mschap/eap-md5
Choose the authentiction method to use. e.g. "-t pap", "-t chap", "-t mschap", or "-t eap-md5",. Defaults to "pap". Using EAP-MD5
requires that the "radeapclient" program is installed.
-x Enables debugging output for the RADIUS client.
-4 Use NAS-IP-Address for the NAS address (default)
-6 Use NAS-IPv6-Address for the NAS address (default)
user Username to send.
password
Password of the user.
radius-server
Hostname or IP address of the radius server. Optionally, you may specify a port by appending :port
nas-port-number
The value of the NAS-Port attribute. Is an integer between 0 and 2^31, and it really doesn't matter what you put here. 10 will do
fine.
secret The shared secret for this client.
ppphint
If you put an integer > 0 here, radtest (or actually radclient) will add the attribute Framed-Protocol = PPP to the request packet.
nasname
If present, this will be resolved to an IP address and added to the request packet as the NAS-IP-Address attribute. If you don't
specify it, the local hostname of the system will be used.
SEE ALSO
radiusd(8), radclient(1).
AUTHOR
Miquel van Smoorenburg, miquels@cistron.nl.
5 April 2010 RADTEST(1)