Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Random web page download wget script
Top Forums Shell Programming and Scripting Random web page download wget script Post 302796841 by Jotne on Sunday 21st of April 2013 01:38:28 AM
Old 04-21-2013
As Scott write, why do you still use back tics `` and expr
You code looks better without.

Code:
#!/bin/bash
url_list=( http://www.bbc.co.uk http://www.cnn.com http://www.msn.com )

#Number of users to mimic simultaneously
users=20

#wget arguments
args="-E -H -T 30 -k -K -p --delete-after --no-cache -e robots=off"

function one_user () {
  local user=$1
  while [ 1 -eq 1 ]; do
    local wait=$((RANDOM % 120 + 30))
    local n=$((RANDOM % 3))
    local url=${url_list[$n]}
    time=$(date +"%T")
    date=$(date +"%m-%d-%y")
    wget=$(wget $args $url 2>&1 | awk '/in/{a=$0}END{print a}' | awk -F "in" '{print$2}')
    echo $date,$time,client$user,$url,$wget
    # echo user = $user wait = $wait url = $url
    sleep $wait
  done
  }

for (( user = 1; user <= $users; user++ )); do
  one_user $user &
done

 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

running shell script thru WEB page ....

....passing variable via list... here 's the HTML code extract : **************** <form method=post action=http://servername/cgi-bin/cgi-comptage_diff.ksh> <table border...........> .............. </table> <table bgcolor=#FFFFFF width="980"> ... (6 Replies)
Discussion started by: Nicol
6 Replies

2. Shell Programming and Scripting

Script to download file using wget

Hi I need a Shell script that will download a text file every second from a http server using wget. Can anyone provide me any pointers or sample scripts that will help me go about this task ??? regards techie (1 Reply)
Discussion started by: techie82
1 Replies

3. Shell Programming and Scripting

how to redirect to a web-page by shell script

Dear all, I am calling a korn shell script(CGI script) by a web-page. This shell script do some checking in a unix file and return true or false. Now within the same script, If it returns true then I want to redirect to another web-page stored in htdocs directory. Example: Login page sends a... (3 Replies)
Discussion started by: ravi18s
3 Replies

4. Shell Programming and Scripting

Perl script to copy contents of a web page

Hi All, Sorry to ask this question and i am not sure whether it is possible. please reply to my question. Thanks in advance. I need a perl script ( or any linux compatible scripts ) to copy the graphical contents of the webpage to a word pad. Say for example, i have a documentation site... (10 Replies)
Discussion started by: anand.linux1984
10 Replies

5. UNIX for Dummies Questions & Answers

Possible to download web page's text to a file?

Hi, Say there is a web page that contains just text only - that is, even the source code is just the text itself, nothing more. An example would be "http://mynasadata.larc.nasa.gov/docs/ocean_percent.txt" Is there a UNIX command that would allow me to download this text and store it in a... (1 Reply)
Discussion started by: Breanne
1 Replies

6. HP-UX

Help running a unix script from a web page

First, let me state that I am completely out of my realm with this. I have a server running HPUX. I'm not even sure if this can be considered a UNIX question and for that let me apologize in advance. I need to create a web page where a client can input 2 variables (i.e. date and phone number).... (0 Replies)
Discussion started by: grinds
0 Replies

7. UNIX for Dummies Questions & Answers

List and download web page files

Hello, Does anyone know of a way to list all files related to a single web page and then to download say 4 files at a time simultaneously until the entire web page has been downloaded successfully? I'm essentially trying to mimic a browser. Thanks. (2 Replies)
Discussion started by: shadyuk
2 Replies

8. Shell Programming and Scripting

Wget - working in browser but cannot download from wget

Hi, I need to download a zip file from my the below US govt link. https://www.sam.gov/SAMPortal/extractfiledownload?role=WW&version=SAM&filename=SAM_PUBLIC_MONTHLY_20160207.ZIP I only have wget utility installed on the server. When I use the below command, I am getting error 403... (2 Replies)
Discussion started by: Prasannag87
2 Replies

9. Shell Programming and Scripting

Refresh web page in bash script

hello, I am trying to refresh my web page which is created in bash script. I have a HTML page which when press a button calls a bash script. this bash script created the same page with dynamic data. When pressing the button I am calling to a function that set time out of 7 seconds and and after... (1 Reply)
Discussion started by: SH78
1 Replies

10. UNIX for Beginners Questions & Answers

How to use cURL to download web page with authentification (form)?

Hello, I'm new in the forum and really beginer, and also sorry form my bad english. I use linux and want to create little program to download automaticaly some pdf (invoices) and put in a folder of my computer. I learn how to do and programme with ubuntu but the program will be implemented... (1 Reply)
Discussion started by: MarcelOrMittal
1 Replies

LEARN ABOUT DEBIAN

bb-datepage.cgi

BB-DATEPAGE.CGI(1)					      General Commands Manual						BB-DATEPAGE.CGI(1)

NAME

       bb-datepage.cgi - Xymon CGI script to view pre-built reports by date

SYNOPSIS

       bb-datepage.cgi?type={day,week,month} --url=URLPREFIX [options]

DESCRIPTION

       bb-datepage.cgi is invoked as a CGI script via the bb-datepage.sh CGI wrapper.

       bb-datepage.cgi	is  passed  a  QUERY_STRING  environment variable with the type of time-selection that is desired: Either "day", "week" or
       "month" can be requested. It will then generate a web form with appropriate day/week/month selection boxes, and based on the users'  selec-
       tion a resulting url is built from the URLPREFIX and the time selection.  The browser is then redirected to this URL.

       The  URL  is  constructed from the URLPREFIX, the type-parameter, the value of the "pagepath" or "host" cookie, and the users' selection as
       follows:

       type=day
	      The final URL is URLPREFIX/daily/YEAR/MONTH/DAY/PAGEPATH.

       type=week
	      The final URL is URLPREFIX/weekly/YEAR/WEEK/PAGEPATH.

       type=month
	      The final URL is URLPREFIX/monthly/YEAR/MONTH/PAGEPATH.

	      YEAR is the full year (4 digits, including century). MONTH is the two-digit number of the month (01..12). DAY is the number  of  the
	      day in the month (01..31). WEEK is the ISO 8601:1988 week-number (01..53). PAGEPATH is the current value of the "pagepath" cookie if
	      set; if it is not set but the "host" cookie is set, then this host is looked up in the bb-hosts file and the page where this host is
	      found is used for PAGEPATH. These two cookies are set by the default web-header templates supplied with Xymon.

OPTIONS

       --url=URLPREFIX
	      This specifies the initial part of the final URL. This option is required.

       --hffile=FILENAME
	      Specifies the template files (from $BBHOME/web/) to use. The default is "--hffile=report".

       --color=COLOR
	      Sets the background color of the generated webpage. The default is blue.

       --env=FILENAME
	      Loads the environment defined in FILENAME before executing the CGI script.

       --debug
	      Enables debugging output.

       $BBHOME/web/report_form_daily
	      HTML form template for the date selection form when type=daily.

       $BBHOME/web/report_form_weekly
	      HTML form template for the date selection form when type=weekly.

       $BBHOME/web/report_form_monthly
	      HTML form template for the date selection form when type=monthly.

       $BBHOME/web/report_header
	      HTML header file for the generated web page

       $BBHOME/web/report_footer
	      HTML footer file for the generated web page

ENVIRONMENT VARIABLES

       BBHOME Used to locate the template files for the generated web pages.

       QUERY_STRING
	      Contains the parameters for the CGI script.

SEE ALSO

       bbgen(1), bb-hosts(5), hobbitserver.cfg(5)

Xymon							    Version 4.2.3:  4 Feb 2009						BB-DATEPAGE.CGI(1)
All times are GMT -4. The time now is 12:02 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy