help/suggestions needed with wget


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting help/suggestions needed with wget
# 1  
Old 01-29-2008
help/suggestions needed with wget

Hi

I'm thinking of using the following command to download some music from websites I visit (designated in the mp3blogs.txt file):

wget -r -l1 -H -t1 -nd -N -np -A.mp3 -erobots=off -i ~/mp3blogs.txt -P ~/Music/WGet

My only question is, is there ANY way to either download files that have been posted ONLY after a certain date (like today) or only download a certain number of files from each site in my txt file?

I know that the -N timestamp will check the files I've downloaded and only download new ones but as far as starting out, I don't want to download ALL the files some sites have available dating back to what could be years. That's why I was wondering about a solution as described above.

I know curl has a -z command that some people say might work but I'm more fond of wget.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

2. Shell Programming and Scripting

WPAR monitoring shell script suggestions needed

Hi All, This is for WPAR monitoring shell script, earlier opened thread was closed, had to open a new thread, as suggested I have used script as below, But am trying to get the output in below format, need suggestions with it. Below is the lswpar output, required output format. ... (7 Replies)
Discussion started by: aix_admin_007
7 Replies

3. UNIX for Advanced & Expert Users

Need suggestions.......

Hello there....i am a final year comp science student.......i am thinking of doing my project on unix platform......which one do u suggest?thanx in advance... (3 Replies)
Discussion started by: theprasad1990
3 Replies

4. Shell Programming and Scripting

Help needed in Curl & Wget

We are trying to invoke a https service from our unix script using curl command. The service is not getting invoked because it is SSL configured. Bypassing certification (using curl –k) does not work. curl -k https://site curl -k -x IP:Port https://site curl -k -x IP:443 https://id:pwd@site ... (0 Replies)
Discussion started by: dineshbabu01
0 Replies

5. Shell Programming and Scripting

Opinions/Ideas/Suggestions needed

I'm currently developing a script to clean out certain directories based on age and name. Part of the assignment is to ensure that the cleaning of a directory is done under the user id of the owner (script is running as root). I have a few ideas on how to do this, but I'd like to hear your... (3 Replies)
Discussion started by: pludi
3 Replies

6. Windows & DOS: Issues & Discussions

Backing up virtual machines - opinions/suggestions needed.

Hi, I was required to do a backup of a virtual machine that runs on vmware. The guest operating system is windows, and the host is windows too. I have to backup the whole directory of the virtual machine (say in linux it'll be in /var/lib/vmware/virtual machines/) to a linux server. Initially... (0 Replies)
Discussion started by: 60doses
0 Replies

7. Solaris

Suggestions Req

Hi all, I have worked on HP UNIX and now i have moved to SunSolaris which i never used to work. I am more on programming side like shell and perl scripting. So i want to know from you experts that i need to take care or changes which i code in sun solaris in compared to HP unix. Suggestions... (1 Reply)
Discussion started by: ravi.sadani19
1 Replies

8. UNIX for Advanced & Expert Users

Looking for Suggestions...

We run WebSphere and by default it wants to install everything under /usr. While I can understand the default (everyone has a /usr) I would like to move this over to a dedicated volume group called apps and then setup my lv's and fs's here. Our WebSphere Admin doesn't like this because apparently... (1 Reply)
Discussion started by: scottsl
1 Replies

9. Filesystems, Disks and Memory

Backup Suggestions.

Hi all, Have two SCO 5.0.5 systems and one Slackware(joy?) system. I've been asked to backup all three systems onto a newly acquired AIT Tape drive that we've installed on one of the SCO boxes. Using the existing cpio backup script on the one SCO works a treat and is really fast (which is... (2 Replies)
Discussion started by: Cameron
2 Replies

10. UNIX for Dummies Questions & Answers

Suggestions on where to begin?

I have been a student at Hendrix Institute for about a year now. My term is comming to an end by the end of december. I have learned varios computer programs for web development that include Flash 5 and Dreamweaver. Actionscripting, Javascript and Database development with Access was all... (4 Replies)
Discussion started by: andrew25008
4 Replies
Login or Register to Ask a Question
Gedcom::WebServices(3pm)				User Contributed Perl Documentation				  Gedcom::WebServices(3pm)

NAME
Gedcom::WebServices - Basic web service routines for Gedcom.pm Version 1.16 - 24th April 2009 SYNOPSIS
wget -qO - http://www.example.com/ws/plain/my_family/i9/name DESCRIPTION
This module provides web service access to a GEDCOM file in conjunction with mod_perl. Using it, A request for imformation can be made in the form of a URL specifying the GEDCOM file to be used, which information is required and the format in which the information is to be delivered. This information is then returned in the specified format. There are currently three supported formats: o plain - no markup o XML o JSON URLs The format of the URLs used to access the web services are: $BASEURL/$FORMAT/$GEDCOM/$XREF/requested/information $BASEURL/$FORMAT/$GEDCOM?search=search_criteria BASEURL The base URL to access the web services. FORMAT The format in which to return the results. GEDCOM The name of the GEDCOM file to use (the extension .ged is assumed). XREF The xref of the record about which information is required. XREFs can be obtained initially from a search, and subsequently from certain queries. requested/information The information requested. This is in the same format as that taken by the get_value method. search_criteria An individual to search for. This is in the same format as that taken by the get_individual method. EXAMPLES
$ wget -qO - 'http://pjcj.sytes.net:8585/ws/plain/royal92?search=elizabeth_ii' /ws/plain/royal92/I52 $ wget -qO - http://pjcj.sytes.net:8585/ws/plain/royal92/I52 0 @I52@ INDI 1 NAME Elizabeth_II Alexandra Mary/Windsor/ 1 TITL Queen of England 1 SEX F 1 BIRT 2 DATE 21 APR 1926 2 PLAC 17 Bruton St.,London,W1,England 1 FAMS @F14@ 1 FAMC @F12@ $ wget -qO - http://pjcj.sytes.net:8585/ws/plain/royal92/I52/name Elizabeth_II Alexandra Mary /Windsor/ $ wget -qO - http://pjcj.sytes.net:8585/ws/plain/royal92/I52/birth/date 21 APR 1926 $ wget -qO - http://pjcj.sytes.net:8585/ws/plain/royal92/I52/children /ws/plain/royal92/I58 /ws/plain/royal92/I59 /ws/plain/royal92/I60 /ws/plain/royal92/I61 $ wget -qO - http://pjcj.sytes.net:8585/ws/json/royal92/I52/name {"name":"Elizabeth_II Alexandra Mary /Windsor/"} $ wget -qO - http://pjcj.sytes.net:8585/ws/xml/royal92/I52/name <NAME>Elizabeth_II Alexandra Mary /Windsor/</NAME> $ wget -qO - http://pjcj.sytes.net:8585/ws/xml/royal92/I52 <INDI ID="I52"> <NAME>Elizabeth_II Alexandra Mary/Windsor/</NAME> <TITL>Queen of England</TITL> <SEX>F</SEX> <BIRT> <DATE>21 APR 1926</DATE> <PLAC>17 Bruton St.,London,W1,England</PLAC> </BIRT> <FAMS REF="F14"/> <FAMC REF="F12"/> </INDI> CONFIGURATION
Add a section similar to the following to your mod_perl config: PerlWarn On PerlTaintCheck On PerlPassEnv GEDCOM_TEST <IfDefine GEDCOM_TEST> <Perl> $Gedcom::TEST = 1; </Perl> </IfDefine> <Perl> use Apache::Status; $ENV{PATH} = "/bin:/usr/bin"; delete @ENV{"IFS", "CDPATH", "ENV", "BASH_ENV"}; $Gedcom::DATA = $Gedcom::ROOT; # location of data stored on server use lib "$Gedcom::ROOT/blib/lib"; use Gedcom::WebServices; my $handlers = [ qw ( plain xml json ) ]; eval Gedcom::WebServices::_set_handlers($handlers); # use Apache::PerlSections; print STDERR Apache::PerlSections->dump; </Perl> PerlTransHandler Gedcom::WebServices::_parse_uri BUGS
Very probably. See the BUGS file. And the TODO file. VERSION
Version 1.16 - 24th April 2009 LICENCE
Copyright 2005-2009, Paul Johnson (paul@pjcj.net) This software is free. It is licensed under the same terms as Perl itself. The latest version of this software should be available from my homepage: http://www.pjcj.net perl v5.14.2 2012-04-12 Gedcom::WebServices(3pm)