Sponsored Content
Top Forums UNIX for Advanced & Expert Users ftp - any best practices for signalling end of transfer? Post 302591544 by _dev_null on Thursday 19th of January 2012 09:53:44 PM
Old 01-19-2012
Question ftp - any best practices for signalling end of transfer?

Hi,

I'm writing shell scripts to handle incoming and outgoing automated sftp transfers between a local server and various remote servers belonging to different organizations.

I'm wondering if there are any recommended or "best practices" for signalling the end of a ftp file transmission (sending multiple files), such as sending an empty ".done" (e.g. "somefilename.done") or similar file, after all other files are sent. Or maybe sending one empty ".done" file per individual data file.

I've seen the ".done" approach here and elsewhere, and have one client who sends a ".eot" (end of transmission) file. I'm just wondering if there are any informal standards or practices for doing this.

The idea being that you don't want the party receiving files to start processing them until it's certain that all expected files have arrived and are complete.

Cheers.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

FTP Transfer Success

Hi I'm using Shell script which will ftp (download) a file from a host. I need to check whether the file has been transfered completley or not. Can anyone tell me how to do that? Thanks Ben (1 Reply)
Discussion started by: abrd600
1 Replies

2. Solaris

ftp transfer

Hi ....I'm trying to ftp 2 pkgs SUNWqlc & SUNWqlcx since they were accidentally deleted from my servers. For one reaseon or the other my server is not loading the cdrom, so I mounted the cdrom on a differernt server & ftp's to it. I tried to get this pkg using the bin & I get an error message... (3 Replies)
Discussion started by: Remi
3 Replies

3. Shell Programming and Scripting

FTP Transfer error

Hi ! I created a similar script as shown below to automate an FTP transfer and encountered errors. Appreciate any advice to resolve this issue. The script: #!/usr/bin/ksh ftp 'IP ADD' << cmd user 'user' 'pass' lcd /home/data get 'FileA' FileB quit cmd Connected to 'IP_ADD'.... (3 Replies)
Discussion started by: cosec
3 Replies

4. Shell Programming and Scripting

file transfer using Ftp

Hi All, I am very new to shell scripting,I have some doubts how we can do a file transfer using ftp I wanted to transfer files using non_secure ftp and secure ftp(sftp) mode, for secure ftp i am using following code verbose="verbose" ftp="/usr/bin/ftp" user="ABC" pass="123"... (1 Reply)
Discussion started by: sudhi
1 Replies

5. Shell Programming and Scripting

File Transfer through FTP

Hi Guys, I wanted to transfer files from FTP machine to Linux Machine. In this, I need to create a batch file that will connect to the Linux Machine and Transfer the files specified as the parameter to the files. Ex : transferfiles Product Time Geography Here transferfiles shld contain... (4 Replies)
Discussion started by: Swapna173
4 Replies

6. Shell Programming and Scripting

FTP (File Transfer)

I am going to transfer file from UNIX directory to remote windows location and i wrote the script but i am getting the error as 'FTP: not found' or i am getting this error 'The file path is not found'. Please help me to resolve this problem as early as possible.(urgent). my script is, ... (6 Replies)
Discussion started by: praka
6 Replies

7. Shell Programming and Scripting

ftp transfer in a for loop

Frdz, i have scenorio like i have to open ftp connection in a for loop and connect to the other server and upload the file again and again till the for loop ends but i am getting problem when i use the following code. for i in `ls ${SOURCE_DIR}/` do FOLDER_NAME=`basename $i` for j in `ls... (4 Replies)
Discussion started by: KiranKumarKarre
4 Replies

8. UNIX for Dummies Questions & Answers

Help with FTP Variable Transfer

Hey all, I'm trying to upload a file to my ftp server through an automated batch program. I will be uploading a file that's name changes according to you IP Address, so I am using the variable %ip% to represent the file's name. The problem occurs when it transfers over to the ftp portion of the... (2 Replies)
Discussion started by: piking
2 Replies

9. Shell Programming and Scripting

[BASH] signalling

Hi guys, I am using slurm to send file to make calculation on a server at my university. The time limit for these calculation is 5 days but sometimes it is not enough. For this reason I need a clean up function that before the calculation ends copy the unfinished calculation file ( in order to... (5 Replies)
Discussion started by: gbengasi
5 Replies

10. UNIX for Beginners Questions & Answers

How to see the status of all the ftp put & get files logs and curent ftp transfer status ?

How to see the status of all the ftp put & get files logs and curent ftp transfer status if any active ftp running in the background ? (2 Replies)
Discussion started by: i4ismail
2 Replies
FTP-UPLOAD(1p)						User Contributed Perl Documentation					    FTP-UPLOAD(1p)

NAME
ftp-upload - batch transfer local files to an FTP server SYNOPSIS
ftp-upload [any-switch]... {[repeatable-switch]... file...}... DESCRIPTION
ftp-upload is used to send local files to an FTP server. It isn't interactive, it's meant to be used from scripts. It is disciplined about its exit value and it doesn't output informational messages by default. There are two kinds of switches. Initial switches have to appear before any filenames, they affect the session as a whole. Repeatable switches can appear interspersed with the file names, they affect the transfer of the files which appear after them on the command line. OPTIONS
Initial switches These have to be used before any file names listed on the command line. --debug Turn debugging on. --help Show the usage message and die. --ignore-quit-failure Don't complain or set a failure exit code just because the QUIT command fails. This can be necessary because some servers, in blatant disregard of RFC 959, close the command channel when you send them an ABOR command. -v, --verbose Print informational messages to stdout. --version Show the version number and exit. Initial switches which specify connection information These also have to be used before any file names listed on the command line. They specify the information used to set up the FTP connec- tion. --account account This specifies the account to be used when logging into the remote system. This is distinct from the user name used to log in. Few systems need this. There is no default. -h, --host host Specify the host to which to connect. There is no default, you have to specify this switch. --passive Force the use of passive (PASV) transfers. Passive transfers are required with some firewall configurations, but if you have such you'd do better to configure Net::FTP so that it knows when to use them (see Net::Config). If you need to use passive transfers with certain (broken) servers, however, this switch is your best bet. Alternatively, you can set $FTP_PASSIVE to 1 in the environment (see Net::FTP). --password pw This gives the password which will be used to login. The default is your email address. Note that you should not specify a real (secret) password this way, as on most systems anybody on the machine can see the arguments you pass to your commands. Use one of other password-setting switches instead. -s, --password-stdin This tells ftp-upload to read the password from standard input. No prompt will be printed, and a single line will be read. Most peo- ple will use this switch to specify the password. Eg, echo 3x9sjJJh | ftp-upload -sh $host -u $user $file Using echo this way is safe where the --password switch isn't if the echo command is built in to the shell. --password-fd fd This is like --password-stdin except that it reads the password from the file descriptor numbered fd. ftp-upload -h $host -u $user --password-fd=3 3<$pw_file $file -u, --user user Specify the user name to use when logging in. The default is "anonymous". Repeatable switches These switches can be used anywhere on the command line (except after the last file name). They affect the transfer of files listed after them. --as remote-name Normally a file is transferred using the same name it has locally. If you use this switch the next file transferred will be called remote-name on the other host instead. ftp-upload --host $host --as index.htm index.html -a, --ascii Perform transfers in ASCII mode. -b, --binary Perform transfers in binary mode. This is the default. -d, --dir dir Change directory to dir on the FTP server before continuing. You can use this multiple times between files, ftp-upload will chdir once for each time you specify it. Using ".." as the dir will cause an FTP "CDUP" to be done rather than a "CWD". --full-path Normally uploaded files go into the current directory on the remote host, even when the local file name given contains slashes. Eg, if you say ftp-upload -h $host /etc/motd ftp-upload will upload the file as motd, not /etc/motd. This differs from how the standard ftp program works, and it also differs with how ftp-upload worked before version 1.3. If you specify --full-path, you'll get the other behavior. A request to upload dir/file will tell the server to store dir/file rather than file. When you use --as the --full-path setting doesn't matter. --full-path only tells the program what name to use when it's choosing the name. --no-full-path Disable --full-path. This is the default. -l, --ls Try to get a remote directory listing of files after transferring them. I say "try" because there's no guaranteed way to do this with the FTP protocol. The command I run is "LIST file". This will generally work if file doesn't contain any special characters. -L, --no-ls Disable the --ls behavior. --tmp-none Transfer files directly, don't do anything special to try to ensure that they don't appear under their real names on the remote machine until the transfer is finished. Each file is transferred with a single simple "STOR". This is the default. --tmp-samedir Transfer files to the remote machine using a temporary name, then rename them when the transfer finishes. This won't work if the remote server doesn't give a recognizable response to the "STOU" command. If the server's response to "STOU" isn't recognized by Net::FTP but is reasonable, Graham Barr might be willing to change Net::FTP to recognize it. If you like you can send the "--debug" output to me and I'll coordinate such requests. --tmp-dir dir Transfer files to dir on the remote host, then rename them when the transfer is complete. This is safer than --tmp-samedir because it doesn't use "STOU" and so it works with more servers. ftp-upload -h $host --tmp-dir incoming $file --tmp-format fmt Transfer files to "sprintf(fmt, file base name)", then rename them when the transfer is complete. Like --tmp-dir, this is safer than --tmp-samedir because it doesn't use "STOU" and so it works with more servers. ftp-upload -h $host --tmp-format tmp.%s $file AUTHOR
Roderick Schertler <roderick@argon.org> perl v5.8.7 2006-03-16 FTP-UPLOAD(1p)
All times are GMT -4. The time now is 09:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy