Web content download


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Web content download
# 1  
Old 03-03-2008
Web content download

There is a web API https://pi.ercot.com/contentproxy/pu...er_id=10001937
As soon as this is fired on the browser a page comes in which many files are listed for downloads
I want to download all the files ....is it possible to write a shell script which can download the files
# 2  
Old 03-04-2008
if you can use Perl, WWW::Mechanize module will help you
Login or Register to Ask a Question

Previous Thread | Next Thread

9 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to use cURL to download web page with authentification (form)?

Hello, I'm new in the forum and really beginer, and also sorry form my bad english. I use linux and want to create little program to download automaticaly some pdf (invoices) and put in a folder of my computer. I learn how to do and programme with ubuntu but the program will be implemented... (1 Reply)
Discussion started by: MarcelOrMittal
1 Replies

2. Shell Programming and Scripting

Wget download file content in unicode

Hi All, I am trying to download a XML from a URL through wget and successful in that but the problem is that I have to check for some special characters inside that XML. But when I download through wget it transfers the content of the XML in plain text and I'm not able to search for those... (2 Replies)
Discussion started by: dips_ag
2 Replies

3. Shell Programming and Scripting

Getting web page content and deliver via mail

Guys could you please help. I want to have shell scripts that get the website html content and same time mail html page to DL. Can any1 help here? (2 Replies)
Discussion started by: AnkitC
2 Replies

4. UNIX for Dummies Questions & Answers

List and download web page files

Hello, Does anyone know of a way to list all files related to a single web page and then to download say 4 files at a time simultaneously until the entire web page has been downloaded successfully? I'm essentially trying to mimic a browser. Thanks. (2 Replies)
Discussion started by: shadyuk
2 Replies

5. Shell Programming and Scripting

Facing issues with Content-Type:application/x-download Content-Disposition:attachment

I am in the process of developing a perl cgi page. I had succeeded in developing the page but there are few errors/issues with the page. description about cgi page: My CGI page retrieves all the file names from an directory and displays the files in drop down menu for downloading the... (5 Replies)
Discussion started by: scriptscript
5 Replies

6. Shell Programming and Scripting

Random web page download wget script

Hi, I've been attempting to create a script that downloads web pages at random intervals to mimic typical user usage. However I'm struggling to link $url to the URL list and thus wget complains of a missing URL. Any ideas? Thanks #!/bin/sh #URL List url1="http://www.bbc.co.uk"... (14 Replies)
Discussion started by: shadyuk
14 Replies

7. Shell Programming and Scripting

Download php content using cli

I'm trying to make a bash script that downloads videos from websites. Is there a cli program for linux that allows me to download php content using one command line entry? I can't use cli curl or wget for this(I think). I'm talking about addresses like these: ... (2 Replies)
Discussion started by: locoroco
2 Replies

8. UNIX for Dummies Questions & Answers

Possible to download web page's text to a file?

Hi, Say there is a web page that contains just text only - that is, even the source code is just the text itself, nothing more. An example would be "http://mynasadata.larc.nasa.gov/docs/ocean_percent.txt" Is there a UNIX command that would allow me to download this text and store it in a... (1 Reply)
Discussion started by: Breanne
1 Replies

9. AIX

check web content - Ksh

Hi everybody! I wonder if I could see some text content of a website, via command line... That could be very helpful, in order to automatize a lot of projects im working on. thx a lot (2 Replies)
Discussion started by: Datinsky
2 Replies
Login or Register to Ask a Question