Sponsored Content
Top Forums Shell Programming and Scripting How to validate files after FTP Post 302450034 by methyl on Wednesday 1st of September 2010 10:23:14 AM
Old 09-01-2010
Just mentioning unix and Windows without flavour or versions is just a bit too vague.

Raw "ftp" is not really suitable for this job but can work well on a good clean private LAN. Beware that many implementations will not work with files larger than 2 Gb or files with fancy filenames.
There is usually much better software than "ftp" but we must assume that you have explored all options.
Even compressing the file prior to transmission gives you a basic verification when uncompressing the file.

Verifying the transfer is more a matter of system design and one would expect a data file to contain checkable data integrity controls.

Always check "ftp" return codes in your script and have provision for timeouts and hangs. It is advisible to transfer a file under a temporary name and only rename the file if the transfer is successful.

I have had variable results when comparing Windows file sizes and unix file sizes for Binary files (it's to do with block padding). The file sizes for Text files will always disagree but the difference can be calculated.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

sftp shell that will validate files

Hi. I've written a shell that will sftp about 5,000 to 10,000 files a day (they are small 200 byte files). Where I'm stuck at is that I need to not only copy those files from the ftp server, I also need to delete the files on the ftp server after they have been ftped BUT before they've been... (0 Replies)
Discussion started by: biglarrrr
0 Replies

2. Shell Programming and Scripting

sftp validate files before removal

I have a shell, which locks calls an expect script that connects to an sftp and downloads all files. I need it to then check each file in my directory and compare the sizes. If the file name and file size match, then delete the file on the server. Is this possible? :confused: (2 Replies)
Discussion started by: tariqjamal
2 Replies

3. Filesystems, Disks and Memory

Not able to FTP the files to a FTP server

Hi , We are facing a weird problem in our project. we need to send some xml & audio files to a remote FTP server from a Linux box, we are doing this in Perl script using Net::FTP->. Issue here is.. when FTPed the files using Perl scripts, only empty files ( 0 byte) are getting created on the... (2 Replies)
Discussion started by: kishorepotta
2 Replies

4. Shell Programming and Scripting

Help needed to validate and merge Two files

Hi.. I am new to unix !!! Please help me in combining two files by validating the first column. The files are tab limited !!! I have a file which has the following contents... DEPT1_BABYCARE 20 abce DEPT2_PHARMACY 50 cdgde DEPT3_WOMENS ... (7 Replies)
Discussion started by: ashokrajaa
7 Replies

5. Shell Programming and Scripting

Validate before moving files.

File Format dataline1,12,,,,; dataline2,24,,,; dataline3,12,,,,; dataline4,24,,,; COUNT=0004; Hi, I have source(/home/rgupta) and destination folders /home/rgupta/arch and /home/rgupta/err. Source folders has many files. I want to move the files from source to destination with some... (1 Reply)
Discussion started by: ravigupta2u
1 Replies

6. Shell Programming and Scripting

[Validate] Script to remove files olderthan X days

Hi, I have written a small shell script to remove the files olderthan X days (say 30). But I not sure how it acts on the filesystem, as I am using rm. if then echo "##############################################" echo "Invalid no .of arguments\n" echo "Usage:\n$0 PATH... (6 Replies)
Discussion started by: karumudi7
6 Replies

7. Shell Programming and Scripting

Validate input files and update

We have a job which we need to run on daily bases, before loading data in a table we need to validate whether the input file is received or not.Inputfile formatsrc_sps_d_Call_Center_Reporting_yyyymmdd_01.dat SPS-Service nameYYYY-yearMM-MonthDD-dayLike above we will get n number of files for... (1 Reply)
Discussion started by: katakamvivek
1 Replies

8. Shell Programming and Scripting

Validate input files daily

We have a job which we need to run on daily bases, before loading data in a table we need to validate whether the input file is received or not. Daily client will place the files in a particular path.Below files which I need to process for 04/01/2013(Load date).... (2 Replies)
Discussion started by: katakamvivek
2 Replies

9. Shell Programming and Scripting

awk to compare files and validate order of headers

The below awk verifies the count and order of each text file in the directory. The script does execute and produce output, however the order of the headers are not compared to key. The portion in bold is supposed to do that. If the order of the headers in each text file is the same as key, then... (0 Replies)
Discussion started by: cmccabe
0 Replies

10. Shell Programming and Scripting

Validate compressed files

Hi All, I have zip file that needs to be validated and checked for 5 times with sleep of 60 seconds. Some thing like below #!/bin/bash counter=1 while do curl -i -k -X GET `strings tmp.txt |grep Location| cut -f2 -d" "` -H "Authorization: Token $TOKEN" -o $zip_file ## this is... (6 Replies)
Discussion started by: Master_Mind
6 Replies
SIMPLEFTP(1)						    InterNetNews Documentation						      SIMPLEFTP(1)

NAME
simpleftp - Rudimentary FTP client SYNOPSIS
simpleftp url [...] DESCRIPTION
simpleftp is a Perl script that provides basic support for fetching files with FTP in a batch oriented fashion. It takes one or more FTP URLs on the command line. The file(s) will be retrieved from the remote server and placed in the current directory with the same basename as on the remote; e.g., <ftp://ftp.isc.org/pub/usenet/CONFIG/active.gz> is stored as active.gz in the current directory. The script properly understands usernames, passwords and ports specified as follows: ftp://user:password@host:port/path/file BUGS
simpleftp is an extremely poor substitute for more complete programs like the freely available wget or ncftp utilities. It was written only to provide elementary support in INN for non-interactive fetching of the files in <ftp://ftp.isc.org/pub/pgpcontrol/> or <ftp://ftp.isc.org/pub/usenet/CONFIG/> without requiring administrators to install yet another package. Its shortcomings as a general purpose program are too numerous to mention, but one that stands out is that downloaded files by simpleftp override existing files with the same name in the local directory. HISTORY
Tossed off by David C Lawrence <tale@isc.org> for InterNetNews. Rewritten to use "Net::FTP" by Julien Elie. $Id: simpleftp.pod 8357 2009-02-27 17:56:00Z iulius $ SEE ALSO
actsync(8). INN 2.5.2 2009-05-21 SIMPLEFTP(1)
All times are GMT -4. The time now is 11:07 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy