UNIX for Dummies Questions & Answers

Hi!

I'm a bit of a unix n00b so be gentle! I need to use ftp to download some files. I have the ftp running in a shell script which had been working perfectly last week. I downloaded around 400 files before it broke on friday.

Now whenever I try to run the script, it starts to download files and then stops arbitrarily in the middle of the first mget. So for example, my mget command would be like:

mget filename*20120320*.csv.gz

With the script changing the date part as appropriate. There are about 8 files per date, and it would maybe download 4 of them before breaking.

Then, for instance, this morning i ran the exact same script and it got 4 dates completed before breaking. It's completely random as to how long it runs for!

I feel at a loss here... Any thoughts on where the problem could lie would be greatly appreciated!

John

---------- Post updated at 12:02 PM ---------- Previous update was at 12:01 PM ----------

Oh forgot to mention - I don't get any error message outputted. It just acts as if it's still doing the mget.

Please mention the Operating System and version of both computers. It is especially important to be specific if Microsoft, IBM or Sun (aka Oracle) are involved.

How big are the files? If small files work, this is very important.

Can you describe the route the data takes from one computer to another, complete with line speeds on each leg and any obstacles like firewalls?
There is a whole difference between downloading via the Internet from Australia and two computers on the same LAN.

All this might save posters asking lots of irrelevant questions.

The most common cause of these symptoms is where one or more or the computers has been set to autonegotiate line speed with a switch and has been re-plugged (or whatever) without a reboot such that the line speed has dropped to say 10 Mbits/sec Half Duplex. The fix is to always hard set LAN speeds on both the server and the network component ... and to never re-plug with the power on!

Other common causes are: cabling fault; hardware fault; WAN fault; duplicate IP address; wonky firewall; bad Name Server; flapping routing; Cisco "Jumbo Packets".

I have fixed this sort of problem and achieved dramatic performance and reliability improvement by lowering the block size to avoid multi-block transfers across mixed-manufacturer network components.

Depends on what you have on your systems.

Methyl, thanks for your quick reply, much appreciated.

A colleague suggested trying wget to see if that would make a difference, and wget seems to be working so far. If it fails again I might post back here, but hopefully not!

Thanks

John

wget is very sturdy and designed for automation. It can even recover from things like a broken connection. If you have it and it works, I'd keep it.