Strange mget problem

Thread Tools Search this Thread
Top Forums UNIX for Dummies Questions & Answers Strange mget problem
# 1  
Old 03-22-2012
Strange mget problem


I'm a bit of a unix n00b so be gentle! I need to use ftp to download some files. I have the ftp running in a shell script which had been working perfectly last week. I downloaded around 400 files before it broke on friday.

Now whenever I try to run the script, it starts to download files and then stops arbitrarily in the middle of the first mget. So for example, my mget command would be like:

mget filename*20120320*.csv.gz

With the script changing the date part as appropriate. There are about 8 files per date, and it would maybe download 4 of them before breaking.

Then, for instance, this morning i ran the exact same script and it got 4 dates completed before breaking. It's completely random as to how long it runs for!

I feel at a loss here... Any thoughts on where the problem could lie would be greatly appreciated!


---------- Post updated at 12:02 PM ---------- Previous update was at 12:01 PM ----------

Oh forgot to mention - I don't get any error message outputted. It just acts as if it's still doing the mget.
# 2  
Old 03-22-2012
Please mention the Operating System and version of both computers. It is especially important to be specific if Microsoft, IBM or Sun (aka Oracle) are involved.

How big are the files? If small files work, this is very important.

Can you describe the route the data takes from one computer to another, complete with line speeds on each leg and any obstacles like firewalls?
There is a whole difference between downloading via the Internet from Australia and two computers on the same LAN.

All this might save posters asking lots of irrelevant questions.

The most common cause of these symptoms is where one or more or the computers has been set to autonegotiate line speed with a switch and has been re-plugged (or whatever) without a reboot such that the line speed has dropped to say 10 Mbits/sec Half Duplex. The fix is to always hard set LAN speeds on both the server and the network component ... and to never re-plug with the power on!

Other common causes are: cabling fault; hardware fault; WAN fault; duplicate IP address; wonky firewall; bad Name Server; flapping routing; Cisco "Jumbo Packets".

I have fixed this sort of problem and achieved dramatic performance and reliability improvement by lowering the block size to avoid multi-block transfers across mixed-manufacturer network components.

Depends on what you have on your systems.

Last edited by methyl; 03-22-2012 at 04:08 PM..
This User Gave Thanks to methyl For This Post:
# 3  
Old 03-22-2012
Methyl, thanks for your quick reply, much appreciated.

A colleague suggested trying wget to see if that would make a difference, and wget seems to be working so far. If it fails again I might post back here, but hopefully not!


# 4  
Old 03-22-2012
wget is very sturdy and designed for automation. It can even recover from things like a broken connection. If you have it and it works, I'd keep it.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

[Solved] Prompt problem while using mget to ftp

I am writing a scritp in which first step is to get some files from a server. I am using mget to do that. here is my FTP code ... HOST="" 28 ftp -inv $HOST <<END &> $FTP_LOG 29 quote USER $USER 30 quote PASS $PASWD 31 cd log 32 prompt off 33 binary 34 mget... (0 Replies)
Discussion started by: shashidhar
0 Replies

2. Shell Programming and Scripting

strange problem

hello all, i am having problem in accessing a directory.I dont think its a permission issue.can anyone help me out. I am using korn sell code: $ ls -ltr sc* lrwxrwxrwx 1 essbase essbase 21 Oct 8 2010 sc_ssp -> /work/nfs/nas2/sc_ssp $ cd sc_ssp ksh: sc_ssp: not found $ (6 Replies)
Discussion started by: manid
6 Replies

3. UNIX for Dummies Questions & Answers

MGET Problem

Hello friends.. I'm new here ate the forum. I came across with it when trying to find a solution for a problem I have and I hope you can help me. I'm an Abap programmer, and I have to make a program in ABAP to copy files from a server to the local computer defined path using FTP commands. ... (5 Replies)
Discussion started by: bleeding_me
5 Replies

4. Shell Programming and Scripting

problem with ftp mget

Hi I am trying to ftp all the files from a directory in server2 to server1 (both unix) The below code is in server1 ftpfile=/home/anitha/ftptxt echo cd mydir >>$ftpfile echo mget *.* >>$ftpfile cat $ftpfile | ftp -i server2 mydir is a directory in server2 and there are some test files in... (4 Replies)
Discussion started by: anijan
4 Replies

5. Shell Programming and Scripting

Strange problem.

Well, my script started off to do what i wanted. Now, i think its not recognizing the pattern so its not moving anything. What i have to do is execute my script command for the move to take effect. So i did that and yayy it worked. Strange thing is that my DESTDIR was empty to begin with.... (2 Replies)
Discussion started by: oxoxo
2 Replies

6. Shell Programming and Scripting

Strange problem

I am using SunOS 5.9 and I don't know why all my commands are getting executed as if an extra 'enter' has been pressed. What could be the reason and how to correct it? Please help. Asty (2 Replies)
Discussion started by: Asty
2 Replies

7. Linux

very strange problem

I have installed Fedora Core on a Toshiba Satellite Pro4600 laptop recently I have experienced a rather mysterious problem if I touch anything specially the keyboard or mouse I see this stuff “67yujhnmyyy” straight away some time it won't stop for while like this... (5 Replies)
Discussion started by: kemobyte
5 Replies

8. UNIX for Dummies Questions & Answers


Hi.. Some of my application were not running properly due to lack of virtual wht i did add one free harddisk as swap file system...and increased the swap memory.. But since than my root file system is showing 100% full thr is no space thr any link between these two..... (1 Reply)
Discussion started by: Prafulla
1 Replies

9. UNIX for Dummies Questions & Answers

Strange Problem

I did an installation of Redhat 7.1 and it seem to install smoothly. I go and reboot and I get the following message: "INIT: Id "x" respawing too fast: disabled for 5 minutes" The screen flashes 4 times every 10 seconds or so. I can't seem to figure why this does this. It might be a simple... (2 Replies)
Discussion started by: forjohnny
2 Replies

10. UNIX for Dummies Questions & Answers

Strange problem.Please Help !

I¡¯m a network operator, mine is an IBM PC server 320, operating system SCO unix 3.2v4.2, triton 3.1 of Baan. Recently, my server went dead every a few hours, no sign & signal shows malfunction suddenly. It looks like a sudden power failure, but the indicator of main power supply is on. Normal... (1 Reply)
Discussion started by: lyhsm
1 Replies
Login or Register to Ask a Question

Featured Tech Videos