Sponsored Content
Top Forums Shell Programming and Scripting Random web page download wget script Post 302796745 by shadyuk on Saturday 20th of April 2013 05:58:24 PM
Old 04-20-2013
Random web page download wget script

Hi,

I've been attempting to create a script that downloads web pages at random intervals to mimic typical user usage. However I'm struggling to link $url to the URL list and thus wget complains of a missing URL. Any ideas?

Thanks

Code:
#!/bin/sh
#URL List
url1="http://www.bbc.co.uk"
url2="http://www.cnn.com"
url3="http://www.msn.com"

#Number of users to mimic simultaneously
users=20

for (( c=1; c<=$users; c++ ))
do 
(
wait=`echo $(( RANDOM% 120 + 30 ))`
rand=`echo $(( RANDOM% 3 + 1 ))`
url=url"$rand"
wget -p $url
sleep $wait
)&
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

running shell script thru WEB page ....

....passing variable via list... here 's the HTML code extract : **************** <form method=post action=http://servername/cgi-bin/cgi-comptage_diff.ksh> <table border...........> .............. </table> <table bgcolor=#FFFFFF width="980"> ... (6 Replies)
Discussion started by: Nicol
6 Replies

2. Shell Programming and Scripting

Script to download file using wget

Hi I need a Shell script that will download a text file every second from a http server using wget. Can anyone provide me any pointers or sample scripts that will help me go about this task ??? regards techie (1 Reply)
Discussion started by: techie82
1 Replies

3. Shell Programming and Scripting

how to redirect to a web-page by shell script

Dear all, I am calling a korn shell script(CGI script) by a web-page. This shell script do some checking in a unix file and return true or false. Now within the same script, If it returns true then I want to redirect to another web-page stored in htdocs directory. Example: Login page sends a... (3 Replies)
Discussion started by: ravi18s
3 Replies

4. Shell Programming and Scripting

Perl script to copy contents of a web page

Hi All, Sorry to ask this question and i am not sure whether it is possible. please reply to my question. Thanks in advance. I need a perl script ( or any linux compatible scripts ) to copy the graphical contents of the webpage to a word pad. Say for example, i have a documentation site... (10 Replies)
Discussion started by: anand.linux1984
10 Replies

5. UNIX for Dummies Questions & Answers

Possible to download web page's text to a file?

Hi, Say there is a web page that contains just text only - that is, even the source code is just the text itself, nothing more. An example would be "http://mynasadata.larc.nasa.gov/docs/ocean_percent.txt" Is there a UNIX command that would allow me to download this text and store it in a... (1 Reply)
Discussion started by: Breanne
1 Replies

6. HP-UX

Help running a unix script from a web page

First, let me state that I am completely out of my realm with this. I have a server running HPUX. I'm not even sure if this can be considered a UNIX question and for that let me apologize in advance. I need to create a web page where a client can input 2 variables (i.e. date and phone number).... (0 Replies)
Discussion started by: grinds
0 Replies

7. UNIX for Dummies Questions & Answers

List and download web page files

Hello, Does anyone know of a way to list all files related to a single web page and then to download say 4 files at a time simultaneously until the entire web page has been downloaded successfully? I'm essentially trying to mimic a browser. Thanks. (2 Replies)
Discussion started by: shadyuk
2 Replies

8. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

9. Shell Programming and Scripting

Refresh web page in bash script

hello, I am trying to refresh my web page which is created in bash script. I have a HTML page which when press a button calls a bash script. this bash script created the same page with dynamic data. When pressing the button I am calling to a function that set time out of 7 seconds and and after... (1 Reply)
Discussion started by: SH78
1 Replies

10. UNIX for Beginners Questions & Answers

How to use cURL to download web page with authentification (form)?

Hello, I'm new in the forum and really beginer, and also sorry form my bad english. I use linux and want to create little program to download automaticaly some pdf (invoices) and put in a folder of my computer. I learn how to do and programme with ubuntu but the program will be implemented... (1 Reply)
Discussion started by: MarcelOrMittal
1 Replies
DJVUSERVE(1)							   DjVuLibre-3.5						      DJVUSERVE(1)

NAME
djvuserve - Generate indirect DjVu documents on the fly. DESCRIPTION
Program djvuserve is a CGI program that can be executed by a HTTP server for serving DjVu documents. This program is able to convert a bundled multi-page document into an indirect document on the fly. USING DJVUSERVE
Program djvuserve must first be installed as a CGI program for your web server. There are several ways to achieve this. The Apache web server, for instance, often defines a specific directory for CGI programs using the ScriptAlias directive. Assume that the file httpd.conf contains the following line: ScriptAlias /cgi-bin/ "/var/www/cgi-bin" It is then sufficient to create a small executable shell script /var/www/cgi-bin/djvuserve containing the following lines: #!/bin/sh exec /full/path/to/djvuserve Suppose that a large bundled multi-page DjVu document is available at the following URL. http://server/dir/doc.djvu The CGI program djvuserve lets you access this same document as an indirect multi-page DjVu document using the following URL. http://server/cgi-bin/djvuserve/dir/doc.djvu/index.djvu Serving indirect multi-page DjVu documents provides for efficiently browsing large document without transferring unnecessary pages over the network. See djvu(1) for more information. Furthermore djvuserve searches certain keywords among the CGI arguments of the URL. The keyword bundled forces serving a bundled document using http://server/cgi-bin/djvuserve/dir/doc.djvu?bundled The keyword download inserts a content disposition HTTP header that suggests to display a save dialog instead of displaying the document. http://server/cgi-bin/djvuserve/dir/doc.djvu?download USING DJVUSERVE AS A HANDLER
The Apache web server provides a way to automatically execute djvuserve for all DjVu documents. This can be achieved using the following directives in either the Apache configuration file or the .htaccess files. Action djvu-server /cgi-bin/djvuserve/ AddHandler djvu-server .djvu Apache then executes program djvuserve for serving all DjVu files. Providing the URL of DjVu file serves this DjVu file as usual, except that bundled multipage documents are converted to indirect documents on the fly. This convenience comes at the expense of the computa- tional cost of executing djvuserve whenever a DjVu file is requested. TECHNICAL DETAILS
Program djvuserve provides a mean to directly access any component of a bundled multi-page DjVu document can be accessed using an extended URL. Suppose that the component file representing page 1 is named p0001.djvu. The following URL provides a direct access to this page: http://server/cgi-bin/djvuserve/dir/doc.djvu/p0001.djvu It is preferred however to access individual pages using the CGI style arguments described in nsdejavu(1), as in the following URL. http://server/cgi-bin/djvuserve/dir/doc.djvu?djvuopts&page=12 The special component file name index.djvu is recognized as a request for the index of the corresponding indirect multi-page document. In fact, when you access a bundled document using djvuserve, the browser gets redirected to the following URL: http://server/cgi-bin/djvuserve/dir/doc.djvu/index.djvu and then behaves as if the bundled file was a directory containing the various component files of an equivalent indirect document. ACCESS CONTROL
Program djvuserve, like many CGI programs, bypasses a number of access protections established in a web server. Assume for instance that your web site contains DjVu files protected by a password. Program djvuserve knows nothing about this protection and will happily serve any DjVu file associated with a valid URL. Access control with djvuserve can be implemented by first remembering that the web server always executes program djvuserve via shell script /var/www/cgi-bin/djvuserve. This script can decide to execute the real program djvuserve on the basis of the target filename available in the environment variable PATH_TRANSLATED. There can be several such scripts providing access to various collections of DjVu files. Each of these scripts can be password protected using the usual methods supported by your web server. KNOWN BUGS
Hyperlinks specified using a relative URL may not work with djvuserve. These URLs are relative to the URL of the DjVu document. Yet djvuserve changes the apparent document URL http://server/dir/doc.djvu into the more complicated URL http://server/cgi-bin/djvuserve/dir/doc.djvu/index.djvu. The extra components change the interpretation of relative URLs. CREDITS
This program was written by Leon Bottou <leonb@users.sourceforge.com>. SEE ALSO
djvu(1), djvmcvt(1), nsdejavu(1) DjVuLibre-3.5 01/22/2002 DJVUSERVE(1)
All times are GMT -4. The time now is 08:58 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy