02-18-2011
176,
5
Join Date: Oct 2008
Last Activity: 11 November 2015, 6:40 PM EST
Location: Orem, Utah
Posts: 176
Thanks Given: 16
Thanked 5 Times in 5 Posts
Web log analysis
I'm looking for good web log analysis software for 95% Apache / 5% IIS logs. There's a lot out there, but our needs are unusual. We're a web hosting company with a large number of VPS servers. For a variety of reasons, we like to develop models of virtual server activity. Among other things, this means obtaining Apache logs from a large number of virtual servers and analyzing them for all sorts of characteristics. Each virtual server maintains its own set of Apache logs. These logs are pulled to a central analysis server and analyzed.
We're not really interested in analyzing in a domain-specific way. We're much more interested in creating composite reports on each of many virtual servers. We can then create a "typical virtual server" model from such reports.
The following would be very useful:
1) The ability to calculate mean and standard deviation of page size. This is far from an exact science, of course, but statistically-reasonable estimates would be nice.
2) Same as 1), but for number of elements per page.
3) Transaction rates, preferably per minute or second.
4) All the other "stuff" that's common to most log analyzers.
Are there any packages that can do what I've described here?