Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Files download using wget
Top Forums Shell Programming and Scripting Files download using wget Post 302763095 by rakesh5300 on Tuesday 29th of January 2013 01:48:05 PM
Old 01-29-2013
Files download using wget
Hi,
I need to implement below logic to download files daily from a URL.

Code:
* Need to check if it is yesterday's file (YYYY-DD-MM.dat)
  * If present then download from URL (sample_url/2013-01-28.dat)
  * Need to implement wait logic if not present 
  * if it still not able to find the file after specific wait then need to send email notification
  * would be great if it can still able to pull 2 files in next day's processing i.e missing file for day before yesterday & yesterday file.
eg: Process date 2013-01-29   files expecting 2013-01-28.dat (but could not find it that day)
Next day it should look for 2 files 2013-01-28.dat & 2013-01-29.dat

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Using wget to download a file

Hello Everyone, I'm trying to use wget recursively to download a file. Only html files are being downloaded, instead of the target file. I'm trying this for the first time, here's what I've tried: wget -r -O jdk.bin... (4 Replies)
Discussion started by: thoughts
4 Replies

2. Shell Programming and Scripting

download a particular file using wget

Hi All I want to download srs8.3.0.1.standard.linux24_EM64T.tar.gz file from the following website : http://downloads.biowisdomsrs.com/srs83_dist/ But this website contains lots of zipped files I want to download the above file only discarding other zipped files. When I am trying the... (1 Reply)
Discussion started by: alphasahoo
1 Replies

3. UNIX and Linux Applications

download file using wget

I need to download the following srs8.3.0.1.standard.linux26_32.tar.gz file from the following website: http://downloads.biowisdomsrs.com/srs83_dist There are many gzip files along with the above one in the above site but I want to download the srs8.3.0.1.standard.linux26_32.tar.gz only from... (1 Reply)
Discussion started by: alphasahoo
1 Replies

4. Shell Programming and Scripting

How to download to a file using wget in perl?

Hi, I want to download some online data using wget command and write the contents to a file. For example this is the URL i want to download and store it in a file called "results.txt". #This is the URL. $url="http://www.example.com"; #retrieve data and store in a file results.txt ... (3 Replies)
Discussion started by: vanitham
3 Replies

5. Shell Programming and Scripting

How to download file without curl and wget

Hi I need a Shell script that will download a zip file every second from a http server but i can't use neither curl nor wget. Can anyone will help me go about this task ??? Thanks!! (1 Reply)
Discussion started by: rubber08
1 Replies

6. Shell Programming and Scripting

How to cancel wget download after 1%?

I am running a video download test and automating that. I wanna know how to stop a wget download session when downloads reached 1% Thanks in advance, Tamil (11 Replies)
Discussion started by: tamil.pamaran
11 Replies

7. Shell Programming and Scripting

Download multiple files uing wget

Need Assistance . Using wget how can i download multiple files from http site. Http doesnt has wild card (*) but FTP has it . Any ideas will be appreciative. wget --timeout=120 --append-output=output.txt --no-directories --cut-dirs=1 -np -m --accept=grib2 -r http://sample.com/... (4 Replies)
Discussion started by: ajayram_arya
4 Replies

8. UNIX for Dummies Questions & Answers

How to download files matching pattern from FTP using CURL or WGET?

Hi, For an order I requested, the provider has uploaded a tar file in public FTP site which internally has tons of files (compressed) and I need to download files that follows particular pattern which would be few hundreds. Note: The order can't be requested for files that follows the... (7 Replies)
Discussion started by: Amalan
7 Replies

9. Shell Programming and Scripting

Wget download file ( do not overwrite )

Hello all, I want to write auto update script for my embedded device, which can check and download newer version of my program and extract the files on the device. The download center is hosted on remote web server . Script checks the hosted file on web site and if the new version is there... (8 Replies)
Discussion started by: stefki
8 Replies

10. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

LEARN ABOUT DEBIAN

dwww-refresh-cache

DWWW-REFRESH-CACHE(8)						      Debian						     DWWW-REFRESH-CACHE(8)

NAME

       dwww-refresh-cache - rebuilds dwww cache directory

SYNOPSIS

       dwww-refresh-cache

DESCRIPTION

       dwww-refresh-cache  is  a  simple  shell  script,  which  deletes  outdated  cache  files and rebuilds contents of the dwww cache directory
       /var/cache/dwww.  In default installation, the script is called from /etc/cron.daily/dwww, so the cache is refreshed every day.

CONFIGURATION VARIABLES

       DWWW_KEEPDAYS
	      Specifies, how many days documents that have not been accessed should be kept in the cache.  Default is 10 days.

       DWWW_QUICKFIND_DB
	      Location	of  the  installed  packages  and  programs  cache  file,  generated  with  help   of	dwww-quickfind(8).    Default	is
	      /var/cache/dwww/quickfind.dat.

       DWWW_DOCBASE2PKG_DB
	      Location	of  the  cache	file,  which  maps installed doc-base files to packages names, used by the dwww-build-menu(8).	Default is
	      /var/cache/dwww/docbase2pkg.dat.

FILES

       /etc/dwww/dwww.conf
	      Configuration file for dwww(7).

       /var/cache/dwww
	      dwww cache directory.

       /etc/cron.daily/dwww.
	      dwww daily cron job

SEE ALSO

       dwww(7), dwww-build-menu(8), dwww-cache(8), dwww-find(8).

AUTHOR

       Robert Luberda.
       See dwww(7), for copyrights and stuff.

dwww 1.11.1							February 15th, 2009					     DWWW-REFRESH-CACHE(8)
All times are GMT -4. The time now is 08:05 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy