Downloading with Wget Post: 302523095

Sponsored Content

Top Forums Shell Programming and Scripting Downloading with Wget Post 302523095 by keltonhalbert on Wednesday 18th of May 2011 12:14:46 AM

05-18-2011

Registered User

Quote:

Originally Posted by KenJackson

I haven't tried this, but I think you could add -nc (or --no-clobber) to prevent it from downloading a file if one already exists, and then truncate rather than delete.

You'll end up with a bunch of zero-length files, but at least they don't use up disk space.

To truncate, this should work: -exec /bin/cp -f /dev/null {} \;

(I use /bin/cp instead of cp because I have cp aliased to "cp -i", and the "-i" overrides the "-f".)

Thanks for the response!
As far as -nc goes, I am not having trouble with it re-downloading the files while they exist, but after they are deleted. Unless of course I am misunderstanding you.

You'll have to pardon my noobishness, but what does -exec /bin/cp -f /dev/null {} \; do exactly? I'm trying learn as I go. I know -exec is for execute, but what is /bin/cp? or -f? or /dev/null? Thanks for your patience.

EDIT: So, I modified the code, and unfortunately wget is now comparing file sizes and re-downloads the data if the sizes do not match up with the directory. Any thoughts?
EDIT2: Got it all figured out. Now I know what -nc is for, and replaced -N with -nc. Thank you!

Last edited by keltonhalbert; 05-18-2011 at 01:32 AM..

keltonhalbert

View Public Profile for keltonhalbert

Find all posts by keltonhalbert

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Downloading

Kindly help us... Is there a way in linux how we'll be able to check which workstation is downloading files from the internet and what file is it? Please put into consideration that our linux has a firewall. Thanks

2. Linux

Downloading Linux

My class "UNIX using Linux" started today. The software that came with my text book will not download. I am running XP pro .. When I load the "guide to UNIX using Linux" disk the only thing that comes up is E:/ showing some files that are on the disk and then it stops and does nothing. Can anyone...

3. Linux

downloading linux

I can't download linux. why? :mad:

4. Solaris

Downloading Solaris 10/08 ?

hi there, Has anybody else had any problems downloading Solaris 10/08 form the Sun web site ? I have tried a number of times, at different times of the day, and have the same results each time. Either the Sun Download manager times out after about 20 retries and I get nothing, or if I...

5. Solaris

HTTP error while downloading solaris patches using wget

Hello, I am getting a HTTP error while downloading solaris patches using wget. 'Downloading unsigned patch 113096-03. --2010-06-18 03:51:15-- http://sunsolve.sun.com/pdownload.pl?target=113096-03&method=h Resolving sunsolve.sun.com (sunsolve.sun.com)... 192.18.108.40 Connecting to...

6. UNIX for Dummies Questions & Answers

Crontab Wget, downloading a file to a FTP

Hi there, I've got my own domain, ftp etc.. I'm using cPanel and I want to download a file periodically, every say 24 hours. I've used this command: wget -t inf http : / / www . somesite . com / webcam.jpg ftp : / / i @ MyDomain . net : Password @ ftp . MyDomain . net^no spaces...

7. UNIX for Dummies Questions & Answers

wget pdf downloading problem

Hi. I am trying to make a mirror of this free online journal: http://www.informaworld.com/smpp/title~content=t716100758~db=all Under the individual issues, the link location for the "Full Text PDF" does not have ".pdf" as an extension -- so when I use wget it misses the file. However clicking...

8. Shell Programming and Scripting

Wget error while downloading from https website

Hi, I would like to download a file from a https website. I don't have the file name as it changes every day. I am using the following command: wget --no-check-certificate -r -np --user=ABC --password=DEF -O temp.txt https://<website/directory> I am getting followin error in my...

9. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403...

10. Shell Programming and Scripting

Wget for downloading a public file (stream) as mp4

I need a hint for using wget for getting a free content from a TV station that is streaming its material for a while until it appears on any video platform, that means no use of illegal methods, because it is on air, recently published and available. But reading the manual for wget I tried the...

LEARN ABOUT NETBSD

paths

PATHS(3)						   BSD Library Functions Manual 						  PATHS(3)

NAME

     paths -- default system paths

SYNOPSIS

     #include <paths.h>

DESCRIPTION

     The <paths.h> header defines some default paths used in NetBSD.  All defined constants are prefixed with _PATH.  The constants include:

	   o   The default user search path set by login(1) and associated utilities such as rshd(8).  This is defined by _PATH_DEFPATH and
	       includes directories such as /usr/bin and /usr/local/bin.

	   o   Default paths for some utilities and device nodes.  Examples include such paths as /dev/null (_PATH_DEVNULL), /dev/mem (_PATH_MEM),
	       and /etc/nologin (_PATH_NOLOGIN), among others.

	   o   Paths for some default directories such as /dev (_PATH_DEV) and /tmp (_PATH_TMP) as well as paths for some miscellaneous utilities
	       such as csh(1), /bin/csh (_PATH_CSHELL).

SEE ALSO

     whereis(1), sysexits(3), types(3), hier(7)

CAVEATS

     The <paths.h> header is specific to NetBSD.

BSD
								   April 8, 2011							       BSD

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Downloading

Discussion started by: eric_hing

2. Linux

Downloading Linux

Discussion started by: Tammy

3. Linux

downloading linux

Discussion started by: buddhika

4. Solaris

Downloading Solaris 10/08 ?

Discussion started by: Freddo_21