Wget recursive with query strings


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Wget recursive with query strings
# 1  
Old 08-24-2010
Java Wget recursive with query strings

I am recursively downloading the archive of our emaillist provider with wget, as our provider is shutting down the listserver soon.

The list is located at https://listserv.surfnet.nl/archives/umpm-bulletin.html

Now I cant make wget save the query-string part of the url, for example https://listserv.surfnet.nl/scripts/wa.cgi?A1=ind1008&L=umpm-bulletin

All it saves is umpm-bulletin.html

Any ideas how to overcome?I tried httrack too, but to no avail.

Last edited by eiland; 08-24-2010 at 08:49 AM.. Reason: layout
# 2  
Old 09-06-2010
bump -> maybe something with curl?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to pass strings from a list of strings from another file and create multiple files?

Hello Everyone , Iam a newbie to shell programming and iam reaching out if anyone can help in this :- I have two files 1) Insert.txt 2) partition_list.txt insert.txt looks like this :- insert into emp1 partition (partition_name) (a1, b2, c4, s6, d8) select a1, b2, c4, (2 Replies)
Discussion started by: nubie2linux
2 Replies

2. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

3. Shell Programming and Scripting

Recursive search on pattern between two strings

Objective: Recursively search all files under a directory for SQL statements that end with ";" Sample input: UPDATE table1 set col=val UPDATE table2 set cola=vala ,colb=valb; UPDATE table3 set col=val Expected output: UPDATE table2 set cola=vala ,colb=valb; (1 Reply)
Discussion started by: krishmaths
1 Replies

4. Shell Programming and Scripting

Shell Script to execute Oracle query taking input from a file to form query

Hi, I need to query Oracle database for 100 users. I have these 100 users in a file. I need a shell script which would read this User file (one user at a time) & query database. For instance: USER CITY --------- ---------- A CITY_A B CITY_B C ... (2 Replies)
Discussion started by: DevendraG
2 Replies

5. Shell Programming and Scripting

Delete lines in file containing duplicate strings, keeping longer strings

The question is not as simple as the title... I have a file, it looks like this <string name="string1">RZ-LED</string> <string name="string2">2.0</string> <string name="string2">Version 2.0</string> <string name="string3">BP</string> I would like to check for duplicate entries of... (11 Replies)
Discussion started by: raidzero
11 Replies

6. UNIX for Dummies Questions & Answers

Delete strings in file1 based on the list of strings in file2

Hello guys, should be a very easy questn for you: I need to delete strings in file1 based on the list of strings in file2. like file2: word1_word2_ word3_word5_ word3_word4_ word6_word7_ file1: word1_word2_otherwords..,word3_word5_others... (7 Replies)
Discussion started by: roussine
7 Replies

7. Shell Programming and Scripting

Query Oracle tables and return values to shell script that calls the query

Hi, I have a requirement as below which needs to be done viz UNIX shell script (1) I have to connect to an Oracle database (2) Exexute "SELECT field_status from table 1" query on one of the tables. (3) Based on the result that I get from point (2), I have to update another table in the... (6 Replies)
Discussion started by: balaeswari
6 Replies

8. UNIX for Dummies Questions & Answers

Help with wget recursive

I want to download every page on a particular website that has: "http://www.website.com/dir/?function=" as the beginning so any page that begins with that will be downloaded, so for example "http://www.website.com/dir/?function=summary&cat=1" and "http://www.website.com/dir/?function=detail&id=1"... (1 Reply)
Discussion started by: guitarscn
1 Replies

9. Shell Programming and Scripting

add the output of a query to a variable to be used in another query

I would like to use the result of a query in another query. How do I redirect/add the output to another variable? $result = odbc_exec($connect, $query); while ($row = odbc_fetch_array($result)) { echo $row,"\n"; } odbc_close($connect); ?> This will output hostnames: host1... (0 Replies)
Discussion started by: hazno
0 Replies

10. Shell Programming and Scripting

How to concatenate two strings or several strings into one string in B-shell?

like connect "summer" and "winter" to "summerwinter"? Can anybody help me? thanks a lot. (2 Replies)
Discussion started by: fontana
2 Replies
Login or Register to Ask a Question