Sponsored Content
Full Discussion: FTp to a server
Top Forums Shell Programming and Scripting FTp to a server Post 302484558 by bisbell on Saturday 1st of January 2011 11:04:31 AM
Old 01-01-2011
I would recommend using scp instead of ftp. The reason is that you can set up a trust relationship using the keys between the two systems and forgo having to script the login.

So, the system with the >10GB file is system 1.
The system that you're moving the files to is system 2.


On System 1:
$ ssh-keygen -t rsa -b 1024
(accept the default file location and don't enter a passphrase)
$ cd ~/.ssh
$ ssh systems2 "mkdir ~/.ssh ; chmod 700 ~/.ssh"
(enter the password for system2)
$ scp id_rsa.pub system2:~/.ssh/authorized_keys
(enter the password for system2)
Now you will be able to ssh and scp from system1 -> system2 without having to give a password. Without going into the details of public key cryptography, this is still a secure connection even though you don't have to give a password. As long as you keep the private key on system1 private you can rely on the security of the connection. It's certainly a lot more secure than scripting a username and password to be sent over an insecure connection like FTP.


Now that you don't have to worry about authentication, you can do a simple find with an scp.
find /FILES -size +20971519 -exec scp {} system2:/directory/path/ \; -exec rm {} \;
This will not recreate the directory structure from system1 onto system2. If you need to recreate the directory structure take a look into rsync.

One thing you'll have to be careful about is transferring and removing a file that is still in the process of being created. If a file is being written and is >10G, but will eventually grow to be 50G, you don't want to start the transfer and delete file from system1 until the file is completely written. Not sure exactly how to deal with that issue, it really boils down to how these files arrive on the first system to begin with.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

how to move a file from one server to another server using FTP in shell script?

I have a file --> file1.txt i need to copy this file to another server using FTP....the 2 servers are server1 and server2..may i know how to write a script that can do this? thanks in advance! Im a newbie to this... (4 Replies)
Discussion started by: forevercalz
4 Replies

2. Shell Programming and Scripting

FTP multiple files from remote server to local server

Hi, I am facing a weired problem in my FTP script. I want to transfer multiple files from remote server to local server everyday, using mget * in my script. I also, want to send an email for successful or failed FTP. My script works for file transfer, but it don't send any mail. There is... (2 Replies)
Discussion started by: berlin_germany
2 Replies

3. Shell Programming and Scripting

ftp script not able to connect to ftp server.

I have the following ftp script to get files from a remote location. However, on running the script I find that I am not even able to connect to ftp server. I am able to connect to ftp server using other GUI ftp tools like WS_FTP using the same IP. IP used here is a dummy IP. What can go... (3 Replies)
Discussion started by: gram77
3 Replies

4. Shell Programming and Scripting

preserving the timestamp of a file when copied from remote server to local server using ftp

Hi, I need to copy few files from remote server to local server. I write a shell script to connect to the remote server using ftp and go to that path. Now i need to copy those files in the remote directory to my local server with the timestamp of all those files shouldnt be changed. ... (5 Replies)
Discussion started by: arunkumarmc
5 Replies

5. Red Hat

when users ftp to server the timezone reflected is UTC but the server is set to TZ in localtime

Guys, Need your help coz my server runs in local time GMT +8, but when client use ftp and login, the resulting timestamp seen in each file is in UTC format. We need to set that the time should be the same as GMT +8 when in ftp session. I am using RHEL 5.3. root@]# ll total 1740... (2 Replies)
Discussion started by: shtobias
2 Replies

6. Windows & DOS: Issues & Discussions

Office server => laptop =>client server ...a lengthy and laborious ftp procedure

Hi All, I need your expertise in finding a way to solve my problem.Please excuse if this is not the right forum to ask this question and guide me to the correct forum,if possible. I am a DBA and on a daily basis i have to ftp huge dump files from my company server to my laptop and then... (3 Replies)
Discussion started by: kunwar
3 Replies

7. HP-UX

[Solved] Unable to rename file in ftp server .Net:FTP perl

Hello All, I am trying to connect to ftp server and get the files. Also i need to rename the file in other ftp dir. rename method is not allowing me to rename the file in other dir. When i tried copy command by using net::FTP:FILE then perl says it is not installed. Can some body help me to... (2 Replies)
Discussion started by: krsnadasa
2 Replies

8. Red Hat

Implement FTP server on RHEL server without using FTP client

We have RHEL 5.8 in our environment, I had a query whether we can implement an FTP server using vsftpd package and Linux configurations like setsebool without using any external FTP clients like FileZilla etc. I am very confused on this. The FTP functionalities that should be present are download &... (3 Replies)
Discussion started by: RHCE
3 Replies

9. Solaris

FTP-ing files from Windows server to UNIX server

I need to transfer files from a Windows server to the Unix server and have to run some shell script on it to get the required output. Is it possible to transfer files from Windows server to unix server through any shell script? If so can you please help me with the details. Thanks in... (8 Replies)
Discussion started by: ssk250
8 Replies

10. Shell Programming and Scripting

Update/Download file from FTP server to UNIX Server

HI Guys, I want to download files from FTP Server to my Unix server. I have tried , buy No Luck . Below Command i have tried. 1-Wget - Error "wget' not found" 2.ftp -n $HOST ...Not Working. 3.scp -i ftp://user:passowrd@hostname:21/ran/on/test.txt Any Suggestion (2 Replies)
Discussion started by: pareshkp
2 Replies
SSH-KEYSIGN(8)						    BSD System Manager's Manual 					    SSH-KEYSIGN(8)

NAME
ssh-keysign -- ssh helper program for host-based authentication SYNOPSIS
ssh-keysign DESCRIPTION
ssh-keysign is used by ssh(1) to access the local host keys and generate the digital signature required during host-based authentication with SSH protocol version 2. ssh-keysign is disabled by default and can only be enabled in the global client configuration file /etc/ssh/ssh_config by setting EnableSSHKeysign to ``yes''. ssh-keysign is not intended to be invoked by the user, but from ssh(1). See ssh(1) and sshd(8) for more information about host-based authen- tication. FILES
/etc/ssh/ssh_config Controls whether ssh-keysign is enabled. /etc/ssh/ssh_host_dsa_key /etc/ssh/ssh_host_ecdsa_key /etc/ssh/ssh_host_rsa_key These files contain the private parts of the host keys used to generate the digital signature. They should be owned by root, read- able only by root, and not accessible to others. Since they are readable only by root, ssh-keysign must be set-uid root if host- based authentication is used. /etc/ssh/ssh_host_dsa_key-cert.pub /etc/ssh/ssh_host_ecdsa_key-cert.pub /etc/ssh/ssh_host_rsa_key-cert.pub If these files exist they are assumed to contain public certificate information corresponding with the private keys above. SEE ALSO
ssh(1), ssh-keygen(1), ssh_config(5), sshd(8) HISTORY
ssh-keysign first appeared in OpenBSD 3.2. AUTHORS
Markus Friedl <markus@openbsd.org> BSD
August 31, 2010 BSD
All times are GMT -4. The time now is 08:53 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy