Sponsored Content
Top Forums Shell Programming and Scripting Using wget and check for non-zero files Post 302141232 by weatherboys on Thursday 18th of October 2007 07:24:28 AM
Old 10-18-2007
Thanks for the quick reply!

So the script will now look like:

Code:
wget --spider -v http://www.test.nl/test.html
wget http://www.test.nl/test.html -O /home/myname/domains/myname.nl/public_html/data/test2.html --quiet

But still gets files if they are zero bytes and overwrites existing files. That's not what I want. I want to check the existence (using your spider option) and I want to allow overwriting in case the files to be copied are not empty.

Can you help me out?!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

wget to check an URL

I all, I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like: while ; do wget --spider <URL>:<port>/console > /dev/null 2>&1 rc=$? done This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies

2. Shell Programming and Scripting

wget skips certain files.

I am trying to use wget to automate downloading of some mp3/wav files. However, I can't get it to follow the link to the mp3s. This is the line (it is not really the website): wget -prl 1 http://website.com/alarmsHowever, if I right-click and copy the link on the webpage in firefox, then... (4 Replies)
Discussion started by: Narnie
4 Replies

3. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

4. Shell Programming and Scripting

Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A... (10 Replies)
Discussion started by: o0110o
10 Replies

5. UNIX for Dummies Questions & Answers

Problem with wget no check certificate.

Hi, I'm trying to install some libraries, when running the makefile I get an error from the "wget --no check certificate option". I had a look help and the option wasn't listed. Anyone know what I'm missing. (0 Replies)
Discussion started by: davcra
0 Replies

6. Shell Programming and Scripting

Check if wget finishes

how can you check if wget finishes? i have this code to store the source of a website url into a variable: source=$(wget --timeout=15 -qO - $site) but this gets stuck sometimes e.g., one site had a virus and it stopped the script how can i check if wget finishes? i tried to run... (5 Replies)
Discussion started by: vanessafan99
5 Replies

7. Shell Programming and Scripting

Files download using wget

Hi, I need to implement below logic to download files daily from a URL. * Need to check if it is yesterday's file (YYYY-DD-MM.dat) * If present then download from URL (sample_url/2013-01-28.dat) * Need to implement wait logic if not present * if it still not able to find the file... (1 Reply)
Discussion started by: rakesh5300
1 Replies

8. Shell Programming and Scripting

Wget - how to ignore files in immediate directory?

i am trying to recursively save a remote FTP server but exclude the files immediately under a directory directory1 wget -r -N ftp://user:pass@hostname/directory1 I want to keep these which may have more files under them directory1/dir1/file.jpg directory1/dir2/file.jpg... (16 Replies)
Discussion started by: vanessafan99
16 Replies

9. Shell Programming and Scripting

Check wget return code

hello check this script it jump to else part, in n both cases, (if files exist or not) wget $MIRROR/kkk.zip && wget $MIRROR/jjj.zip RC="$?" if ] then echo -e "$RED Ooops, Download links are broken...! $RESET" else echo -e "$GREEN Everything is fine, Cheers ... $RESET" fi (4 Replies)
Discussion started by: nimafire
4 Replies

10. UNIX for Advanced & Expert Users

CMD to check status of the server using Wget

Hi All, Using Wget I'm able to get the status of the server.....only when the server is completely down or up.... but problem here in script is Suppose if the server got hang I mean to say that if the server is taking long time to login, for example normally the server takes 3 seconds to login... (3 Replies)
Discussion started by: manohar2013
3 Replies
wibble-test-genrunner(1)				      General Commands Manual					  wibble-test-genrunner(1)

NAME
wibble-test-genrunner - Code generator for wibble testsuites. SYNOPSIS
wibble-test-genrunner header <file> wibble-test-genrunner main <files> DESCRIPTION
The program generates .cpp files that are then compiled and linked into a test program for running tests from .test.h files. Examples of such .test.h files may be found among wibble headers, eg. /usr/include/wibble/regexp.test.h. In the first form, the program processes a single header file (usually of the form foo.test.h) and produces a corresponding .cpp file to be compiled. In the second form, it takes as <files> all the .test.h headers and produces a single main.cpp which contains the main() function of the test program, which then runs all the tests in all the .test.h files. The generated source code is always written to standard output and it is left up to the user to redirect it to a meaningful location. The program currently has no options other than the two forms above. The program is intended to be run as part of build process of programs or libraries using the wibble testing framework. For convenient use from CMake, there is a test.cmake script under /usr/share/wibble, that takes care of producing all the .cpp files (both per-header and the main one), compiling them and linking them into a single binary which executes the testsuite. AUTHOR
Petr Rockai <me@mornfall.net> wibble-test-genrunner(1)
All times are GMT -4. The time now is 10:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy