Sponsored Content
Top Forums Shell Programming and Scripting Using wget and check for non-zero files Post 302141232 by weatherboys on Thursday 18th of October 2007 07:24:28 AM
Old 10-18-2007
Thanks for the quick reply!

So the script will now look like:

Code:
wget --spider -v http://www.test.nl/test.html
wget http://www.test.nl/test.html -O /home/myname/domains/myname.nl/public_html/data/test2.html --quiet

But still gets files if they are zero bytes and overwrites existing files. That's not what I want. I want to check the existence (using your spider option) and I want to allow overwriting in case the files to be copied are not empty.

Can you help me out?!
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

wget to check an URL

I all, I wrote an script which starts a Weblogic server and waits until its loaded to deploy several apps. The way I checked was something like: while ; do wget --spider <URL>:<port>/console > /dev/null 2>&1 rc=$? done This works perfectly because it's an HTML site and when server is... (2 Replies)
Discussion started by: AlbertGM
2 Replies

2. Shell Programming and Scripting

wget skips certain files.

I am trying to use wget to automate downloading of some mp3/wav files. However, I can't get it to follow the link to the mp3s. This is the line (it is not really the website): wget -prl 1 http://website.com/alarmsHowever, if I right-click and copy the link on the webpage in firefox, then... (4 Replies)
Discussion started by: Narnie
4 Replies

3. Shell Programming and Scripting

how to limit files downloaded by wget

I am trying to download a page and retrieve only wav and mp3 files via wget. the website is: Alarm Sounds | Free Sound Effects | Alarm Sound Clips | Sound Bites my command is : wget -rl 2 -e robots=off -A wav,mp3 http://soundbible.com/tags-alarm.html When not using the -A wav,mp3... (2 Replies)
Discussion started by: Narnie
2 Replies

4. Shell Programming and Scripting

Help with WGET and renaming downloaded files :(

Hi everybody, I would greatly appreciate some expertise in this matter. I am trying find an efficient way to batch download files from a website and rename each file with the url it originated from (from the CLI). (ie. Instead of xyz.zip, the output file would be http://www.abc.com/xyz.zip) A... (10 Replies)
Discussion started by: o0110o
10 Replies

5. UNIX for Dummies Questions & Answers

Problem with wget no check certificate.

Hi, I'm trying to install some libraries, when running the makefile I get an error from the "wget --no check certificate option". I had a look help and the option wasn't listed. Anyone know what I'm missing. (0 Replies)
Discussion started by: davcra
0 Replies

6. Shell Programming and Scripting

Check if wget finishes

how can you check if wget finishes? i have this code to store the source of a website url into a variable: source=$(wget --timeout=15 -qO - $site) but this gets stuck sometimes e.g., one site had a virus and it stopped the script how can i check if wget finishes? i tried to run... (5 Replies)
Discussion started by: vanessafan99
5 Replies

7. Shell Programming and Scripting

Files download using wget

Hi, I need to implement below logic to download files daily from a URL. * Need to check if it is yesterday's file (YYYY-DD-MM.dat) * If present then download from URL (sample_url/2013-01-28.dat) * Need to implement wait logic if not present * if it still not able to find the file... (1 Reply)
Discussion started by: rakesh5300
1 Replies

8. Shell Programming and Scripting

Wget - how to ignore files in immediate directory?

i am trying to recursively save a remote FTP server but exclude the files immediately under a directory directory1 wget -r -N ftp://user:pass@hostname/directory1 I want to keep these which may have more files under them directory1/dir1/file.jpg directory1/dir2/file.jpg... (16 Replies)
Discussion started by: vanessafan99
16 Replies

9. Shell Programming and Scripting

Check wget return code

hello check this script it jump to else part, in n both cases, (if files exist or not) wget $MIRROR/kkk.zip && wget $MIRROR/jjj.zip RC="$?" if ] then echo -e "$RED Ooops, Download links are broken...! $RESET" else echo -e "$GREEN Everything is fine, Cheers ... $RESET" fi (4 Replies)
Discussion started by: nimafire
4 Replies

10. UNIX for Advanced & Expert Users

CMD to check status of the server using Wget

Hi All, Using Wget I'm able to get the status of the server.....only when the server is completely down or up.... but problem here in script is Suppose if the server got hang I mean to say that if the server is taking long time to login, for example normally the server takes 3 seconds to login... (3 Replies)
Discussion started by: manohar2013
3 Replies
CDDB-TOOL(1)						      General Commands Manual						      CDDB-TOOL(1)

NAME
cddb-tool - Get CDDB information SYNOPSIS
cddb-tool parse file [-i|-d|-a|-t] [num|-c|-p] cddb-tool template disc-id tracks cddb-tool send [file] address cddb-tool query server user host disc-id tracks cddb-tool read server user host disc-id genre cddb-tool help DESCRIPTION
cddb-tool is a backend tool for abcde. It is passed information from cd-discid and uses it look up the CD title, artist, and track informa- tion on the cddb database (http://freedb.freedb.org) on the internet. COMMANDS
parse [file] option Get data out of a cddb file (use "-" for stdin). Options (use one and only one): id prints disc id album prints album title track num prints title of track num artist prints artist category prints category cddbgenre synonym for category all parse file and dumps to stdout in a form sourceable by the shell send [file] address Mails file file (or stdin of no file specified) to specified address, using correct format. Category should be one of blues, classical, country, data, folk, jazz, newage, reggae, rock, soundtrack, misc. template [disc-id] [tracks] Generates a template (empty) cddb file to stdout. The command line should be essentially the output of cd-discid. query [server] [user] [host] [disc-id] [tracks] Looks up disc on server (should be of form "http://host/cddb/cddb.cgi") remainder of command line is in the same form as that returned by the cd-discid program. Uses wget, so if you need to use a proxy then just configure wget to do so. user and host will be used for identify- ing ourselves to the CDDB server. Outputs response code and entries with matching CDDB entries, if successful, from the CDDB server. read [server] [user] [host] [disc-id] [genre] Dumps CDDB entry to stdout. File will contain an extra #CATEGORY= line, which leaves it a valid CDDB file but which will be recognized by parse and send commands. Uses wget, so if you need to use a proxy then just configure wget to do so. user and host will be used for identi- fying ourselves to the CDDB server. help Prints usage information. AUTHOR
Nathaniel Smith <njs@uclink4.berkeley.edu>, maintained as part of abcde by Robert Woodcock <rcw@debian.org>. Manual page compiled from help information by Robert Woodcock. CDDB-TOOL(1)
All times are GMT -4. The time now is 09:52 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy