Web log analysis

 
Thread Tools Search this Thread
Special Forums UNIX and Linux Applications Web log analysis
# 1  
Old 02-18-2011
Web log analysis

I'm looking for good web log analysis software for 95% Apache / 5% IIS logs. There's a lot out there, but our needs are unusual. We're a web hosting company with a large number of VPS servers. For a variety of reasons, we like to develop models of virtual server activity. Among other things, this means obtaining Apache logs from a large number of virtual servers and analyzing them for all sorts of characteristics. Each virtual server maintains its own set of Apache logs. These logs are pulled to a central analysis server and analyzed.

We're not really interested in analyzing in a domain-specific way. We're much more interested in creating composite reports on each of many virtual servers. We can then create a "typical virtual server" model from such reports.

The following would be very useful:

1) The ability to calculate mean and standard deviation of page size. This is far from an exact science, of course, but statistically-reasonable estimates would be nice.
2) Same as 1), but for number of elements per page.
3) Transaction rates, preferably per minute or second.
4) All the other "stuff" that's common to most log analyzers.

Are there any packages that can do what I've described here?
# 2  
Old 02-18-2011
Maybe AWStats ?
(not sure about the page size deviation)

Also JMT (initially designed to handle java log ... but ... who can handle a java log can just handle any log Smilie)
See what it can do

Last edited by ctsgnb; 02-18-2011 at 03:37 PM..
Login or Register to Ask a Question

Previous Thread | Next Thread

6 More Discussions You Might Find Interesting

1. OS X (Apple)

Web Request log

Hey all, I would like to track what applications are requesting access at the unix level. is there a log or a way to capture all network access requests? Not with applications. (1 Reply)
Discussion started by: Phorn
1 Replies

2. Shell Programming and Scripting

Log Analysis with AWK with Time difference

I would like to write a shell script that calculated the time difference bettween the log entries. If the time difference is higher as 200 sec. print the complette lines out. My Problem is, i am unable to jump in the next line and calculate the time difference. Thank you for your Help. ... (5 Replies)
Discussion started by: fabian3010
5 Replies

3. Homework & Coursework Questions

Parse a Web Server Access Log

1. The problem statement, all variables and given/known data: Write a parser for a web server access log that will provide the statistics outlined below. Remember to format your output in a neat form. You may complete this assignment with one Awk script or a shell script using a combination of... (6 Replies)
Discussion started by: codyhazelwood
6 Replies

4. Shell Programming and Scripting

Log File Analysis

Hi All, Given a list of date and time, how can i split them in intervals of 1 hour. The input file is listed as below f.e.: 16-OCT-2009 09:11:47 10.65.4.24. . . (1 Reply)
Discussion started by: x-plicit78
1 Replies

5. UNIX for Advanced & Expert Users

WEB Server Log File Analysis using awk/sed/grep

I'm trying to find a way to show large page sizes (page size in K) from multiple web server log files. Essentially I want to show only rows from a file where a specific column is larger than some value. Has anyone ever done this type of log analysis? If so, a snippet of code would be very... (2 Replies)
Discussion started by: mike_cataldo@ad
2 Replies

6. Shell Programming and Scripting

Problems with Perl/KSH Web Log Script

Hi, I am writing a series of scripts for work to analyse intranet access logs. All of the scripts do as they should when run individually from the shell, but only when run from certain directories. This sounds like it may be a PATH issue but I am not sure. When I run a certain script, say... (3 Replies)
Discussion started by: mmanders
3 Replies
Login or Register to Ask a Question