Sponsored Content
Top Forums UNIX for Advanced & Expert Users Best way to transfer files to remote FTPS server instead of local FTPS server Post 302999132 by waavman on Tuesday 13th of June 2017 07:16:00 PM
Old 06-13-2017
Best way to transfer files to remote FTPS server instead of local FTPS server

Hi,

I am working on an application which runs on an Informatica Red-Hat 5.10 Linux Server.
The application involves several Informatica ETL workflows which generate 100s of Text files with lot of data. Many of the files will each be up to 5 GB in size.
Currently the Informatica server itself acts as an FTPS server and so these files are stored on the local file system by the ETL workflows and my application's downstream consumer applications use this Informatica server as their FTPS server and pull these data files from this server to their local client machines.
But going forward, due to space limitations and performance issues on Informatica server we are planning on acquiring a new Redhat Linux FTPS server and have these data files (which are gigabytes in size) generated by the ETL workflows on the local Informatica servers, pushed to the new remote FTPS server instead of storing them locally. Then I will have all downstream consumers connect to the new remote FTPS server to pull the data files they need. So then the informtica server will become a purely app server and no longer have the overhead of being an FTPS server
My question is what do you think is the best approach/strategy you would suggest for me to have the huge data files generated locally on my Informatica server shipped/transferred automatically to the new remote Linux FTPS server everytime they are generated. Since these files are huge, I know there will be latency in pushing the files from local Informatica server to remote FTPS server across network unlike before when they were being stored locally. Which Unix strategy or idea do you think would be best in minimizing these latency issues in shipping these huge files across the network to the remote FTPS server or not impact the SLAs' for file availability on the FTPS server too significantly so that downstream consumers applications do not see significant delays in file availability times. Lftp, rsync are few of the tools I have used. But are good enough for this purpose or is there some more creative approach that I could use. I know slight delays are inevitable since there is always a difference between storing files locally versus shipping them across the network to a remote FTPS server. But I am curious as to which approach will minimize this latency. Any inputs/ideas would be greatly appreciated.

thanks
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

FTP multiple files from remote server to local server

Hi, I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies

2. HP-UX

Transfer file from local unix server to remote server

want to remove the thread thanks (2 Replies)
Discussion started by: indira
2 Replies

3. Shell Programming and Scripting

Transfer file from local unix server to remote server

want to remove this thread. thanks (2 Replies)
Discussion started by: indira
2 Replies

4. UNIX for Dummies Questions & Answers

compare zip files from a local to remote server

Good evening I need your help pease I know there are 2 commands(diff, or cp) to compare files in a directory. but the question arises: 1. can i compare zip files or ive got to unzip them? 2. how can i compare 2 files from a local to a remote server? (is there any special commad or ive got... (4 Replies)
Discussion started by: alexcol
4 Replies

5. UNIX for Dummies Questions & Answers

How to copy files from remote server to local?

Hi experts, I 'm newbie to unix world, now I have task to copy the latest files from remote server to my local. I believe this must be very common request in this community. I want you do it one more time for me please. My requirement is something like this: I receive files in the below... (3 Replies)
Discussion started by: parpaa
3 Replies

6. Solaris

Script to get files from remote server to local server through sftp without prompting for password

Hi, I am trying to automate the process of fetching files from remote server to local server through sftp. I have the username and password for the remote solaris server. But I need to give password manually everytime i run the script. Can anyone help me in automating the script such that it... (3 Replies)
Discussion started by: ssk250
3 Replies

7. UNIX for Dummies Questions & Answers

Lftp for a remote server which uses FTPS

Hi All, I am new to SHell scripting, can someone please help me with the below requirement. 1) LFTP a file to a remote server which supports FTPS. My current enviroment is Sun Solaris 5.10 2) I need to incorporate this in a shell which is currently sending files to a server that accepts... (1 Reply)
Discussion started by: subbu
1 Replies

8. Shell Programming and Scripting

Do I require remote login access to a windows server to transfer files from a UNIX server

Hi All I need to transfer a file from a UNIX server to a windows server. I saw that it is possible to do this using scp command by looking at the forum listed below: ... (2 Replies)
Discussion started by: vx04
2 Replies

9. Shell Programming and Scripting

Shell scripting to transfer files from one to another server through ftps

Hi Guyz ,, I'm an ERP functional guy , I really need to automate a script which required a shell script but have a little knowledge in shell scripting. I need my generated files to be zipped first in one directory lets say (xyz) and then it needs to transfer another ftp server in... (3 Replies)
Discussion started by: Shogundion
3 Replies

10. Shell Programming and Scripting

File transfer using FTPS(SSL)

Hi Team, I am currently working on an integration project where in we have planned to receive files from an external source onto our system via FTPS(SSL). I am new to this part and would like to know the points for consideration and the points to learn to get this done. The files we receive from... (1 Reply)
Discussion started by: Rads
1 Replies
updmv(8)						      System Manager's Manual							  updmv(8)

NAME
updmv - Moves customized, user, and inventory data files to or from storage. SYNOPSIS
/usr/bin/updmv -i location /usr/bin/updmv -o location OPTIONS
Copies the Customized File Inventory (CFI) and User File Inventory (UFI) files and those files listed in these inventory files from the specified location to the /var/adm/install/update directory. Copies the CFI and UFI files and those files listed in the inventory files to the specified location. OPERANDS
Specifies the storage area of the files. This area can be a tape device, disk device, or a Remote Installation Services (RIS) server area. You replace the location operand with the name of the tape or disk device, or the server name for a RIS server area. For tape stor- age, you must use a no-rewind tape device such as /dev/ntape/tape0_d0. For disk storage, specify the complete directory path. For storage in a RIS server area, you must register the client in the server's RIS database. When you specify the server, the server name must be followed by a colon (:). DESCRIPTION
The updmv utility moves files listed in the User File Inventory (UFI) file and the Customized File Inventory (CFI) file to or from storage. The UFI file, created by the genufi utility, identifies user files created after you installed the operating system. The CFI file, created by the invsync utility, identifies files that were customized after the installation. You must have superuser privileges to run the updmv utility. During a new installation, some user files and customized files can be overwritten. You can identify and save files from the current installation environment for later integration into the new installation environment. Use the updmv utility to move these files to a stor- age area or from the storage area to the update area. To store the files listed in the CFI and UFI files, invoke the updmv utility with the -o option followed by the location operand. The sys- tem must be shut down to single-user mode before you specify the updmv utility with the -o option. To make these files accessible for integration into a new installation environment, invoke the updmv utility with the -i option followed by the location operand. The updmv utility moves the listed files to the /var/adm/install/update directory so that they can be merged into the new installation environment with the updmerge utility. EXAMPLES
Save the files to a tape device: # updmv -o /dev/ntape/tape0_d0 You must use a no-rewind tape device. Store the files on a disk device: # updmv -o /mnt/mydata Store the files in a RIS server area: # updmv -o midvale: To restore these files, replace the -o option with the -i option. Restore the files that were stored in a RIS server area: # updmv -i midvale: The restored image resides in /var/adm/install/update. RESTRICTIONS
You must have superuser privileges to run the updmv utility. The system must be in single-user mode before you invoke the updmv utility with the -o option. FILES
Customized File Inventory User File Inventory SEE ALSO
genufi(8), invsync(8), updmerge(8) updmv(8)
All times are GMT -4. The time now is 10:40 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy