A script needs to be created to collect all HTTP GET requests containing a particular string
Hi friends,
A script needs to be created to collect all HTTP GET requests containing a particular string say abcd.gif in the url path along with the IP address of the client that issued the request. The source of this data is the web server logs. Also Each script execution should extract client IP address and timestamp and record this either in a file or in a DB.
Logs are like
and capture IP address 10.254.17.140 and timestamp 13/Jul/2012:05:20:49 in a file
Any help will be greatly appreciated
Regards,
Surendra
Moderator's Comments:
Please use code tags next time for your code and data.
Last edited by zaxxon; 07-23-2012 at 10:54 AM..
Reason: code tags, see PM
On a sparc solaris 8 host running sunone webserver 6 I would like to limit the http requests that can be used when port 80 is accessed. We currently have http/1.0 enabled.
For example I would like to remove the http request DELETE.
Regards,
BLP (1 Reply)
Hi folks,
I am trying to build this GUI application that will perform some http requests to a specific server.
Basically I will use curl to "pretend" that it is a browser a sending an http request from a form.
My http knowledge is very low, and the site is tricky, but I think if I could... (3 Replies)
How do i collect Date & Time from Different Directories in a script
The script iam using for a single directory is :
ls -l | grep awk '{print $8}' (2 Replies)
My management is looking for me to sweep my Unix servers and report on (1) how much SAN space has been allocated to each server and (2) how much is used (defined as having a file on those bits). Basically stuff one can get from "df", but they just want SAN totals (not local disks). NAS is exempt.
... (1 Reply)
Hi, I'm newbie in Korn scripting as well as in Unix. I need to have a script to do the following.
1) ssh multiple hosts
-In my case, login credentials are not an issue as all keys authorized by default. Example of host IP (24.35.14.6, 24.35.14.7, 24.35.14.8, 24.35.14.9, ... (2 Replies)
Environmnet: HP-UX B.11.31 U ia64
RDBMS: Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
Question:
When server hangs or node evicts, we open up tickets with Oracle Support and Oracle Support ask for some list of log files.
Currently we can not use the... (2 Replies)
Hi,
I'm trying to write a script to determine the time gap between HTTP PUT and HTTP DELETE requests in the HTTP Servers access log.
Normally client will do HTTP PUT to push content e.g. file_1.txt and 21 seconds later it will do HTTP DELETE, but sometimes the time varies causing some issues... (3 Replies)
I am trying to collect logs from a system in expert mode. Where I login to system and then change mode to expert mode and issue command to collect the log. I have a script which works but It waits for 30 mins even though log collection is complete before that. I have two issues with existing... (4 Replies)
Discussion started by: shunya
4 Replies
LEARN ABOUT DEBIAN
http::recorder
HTTP::Recorder(3pm) User Contributed Perl Documentation HTTP::Recorder(3pm)NAME
HTTP::Recorder - record interaction with websites
SYNOPSIS
Using HTTP::Recorder as a Web Proxy
Set HTTP::Recorder as the user agent for a proxy, and it rewrites HTTP responses so that additional requests can be recorded.
The Proxy Script
For quick start, run the httprecorder script
httprecorder
This will open a local proxy on port 8080, and will dump the recorded traffic to a file named http_traffic in the current directory. use
the -help parameter for usage info
Start the proxy script, then change the settings in your web browser so that it will use this proxy for web requests. For more information
about proxy settings and the default port, see HTTP::Proxy.
The script will be recorded in the specified file, and can be viewed and modified via the control panel.
For better control, use this example:
#!/usr/bin/perl
use HTTP::Proxy;
use HTTP::Recorder;
my $proxy = HTTP::Proxy->new();
# create a new HTTP::Recorder object
my $agent = new HTTP::Recorder;
# set the log file (optional)
$agent->file("/tmp/myfile");
# set HTTP::Recorder as the agent for the proxy
$proxy->agent( $agent );
# start the proxy
$proxy->start();
Start Recording
Now you can use your browser as your normally would, and your actions will be recorded in the file you specified. Alternatively, you can
start recording from the Control Panel.
Using the Control Panel
If you have Javascript enabled in your browser, go to the HTTP::Recorder control URL (http://http-recorder by default), optionally type a
URL into the "Goto page" field, and click "Go".
In the new window, interact with web sites as you normally do, including typing a new address into the address field. The Control Panel
will be updated after each recorded action.
The Control Panel allows you to modify, delete, or save your script.
SSL sessions
As of version 0.03, HTTP::Recorder can record SSL sessions.
To begin recording an SSL session, go to the control URL (http://http-recorder/ by default), and enter the initial URL. Then, interact
with the web site as usual.
Script output
By default, HTTP::Recorder outputs WWW::Mechanize scripts.
However, you can override HTTP::Recorder::Logger to output other types of scripts.
Functions
new
Creates and returns a new HTTP::Recorder object, referred to as the 'agent'.
$agent->prefix([$value])
Get or set the prefix string that HTTP::Recorder uses for rewriting responses.
$agent->control([$value])
Get or set the URL of the control panel. By default, the control URL is 'http-recorder'.
The control URL will display a control panel which will allow you to view and edit the current script.
$agent->logger([$value])
Get or set the logger object. The default logger is a HTTP::Recorder::Logger, which generates WWW::Mechanize scripts.
$agent->ignore_favicon([0|1])
Get or set ignore_favicon flag that causes HTTP::Recorder to skip logging requests favicon.ico files. The value is 1 by default.
$agent->file([$value])
Get or set the filename for generated scripts. The default is '/tmp/scriptfile'.
Bugs, Missing Features, and other Oddities
Javascript
WWW::Mechanize can't play back Javascript actions, and HTTP::Recorder doesn't record them.
Why are my images corrupted?
HTTP::Recorder only tries to rewrite responses that are of type text/*, which it determines by reading the Content-Type header of the
HTTP::Response object. However, if the received image gives the wrong Content-Type header, it may be corrupted by the recorder. While
this may not be pleasant to look at, it shouldn't have an effect on your recording session.
See Also
See also LWP::UserAgent, WWW::Mechanize, HTTP::Proxy.
Requests & Bugs
Please submit any feature requests, suggestions, bugs, or patches at http://rt.cpan.org/, or email to bug-HTTP-Recorder@rt.cpan.org.
If you're submitting a bug of the type "X doesn't record correctly," be sure to include a (preferably short and simple) HTML page that
demonstrates the problem, and a clear explanation of a) what it does that it shouldn't, and b) what it should do instead.
Author
Copyright 2003-2005 by Linda Julien <leira@cpan.org>
Maintained by Shmuel Fomberg <semuelf@cpan.org>
Released under the GNU Public License.
perl v5.14.2 2012-04-23 HTTP::Recorder(3pm)