Login or Register to Ask a Question and Join Our Community

Sponsored Content
Full Discussion: Apache log file pharsing, need help!!!!
Top Forums Shell Programming and Scripting Apache log file pharsing, need help!!!! Post 302142038 by summer_cherry on Wednesday 24th of October 2007 03:37:43 AM
Old 10-24-2007
awk
Hi,

A little complex, but it should be ok. I have tested it on solaris.

input:
Code:
88.118.118.214 - - [22/Oct/2007:20:15:13 +0300] "GET /~tomasa/kg/data/uzd.html HTTP/1.1" 304 - "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)"
text1 text2 text3 [text4] "text5" text6 text7 "text8" "text9"

output:
Code:
field1= 88.118.118.214
field2= -
field3= -
field4= 22/Oct/2007:20:15:13 +0300
field5= GET /~tomasa/kg/data/uzd.html HTTP/1.1
field6= 304
field7= -
field8= -
field9= Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322)
field1= text1
field2= text2
field3= text3
field4= text4
field5= text5
field6= text6
field7= text7
field8= text8
field9= text9

code:
Code:
nawk '
function getVal(str,pos,sta,end)
{
	for(i=1;i<=pos;i++)
	{
		str=substr(str,index(str,sta)+1)
	}
	a=substr(str,1,index(str,end)-1)
	return a
}
{
print "field1= "$1
print "field2= "$2
print "field3= "$3
print "field4= "getVal($0,1,"[","]")
print "field5= "getVal($0,1,"\"","\"")
print "field6= "getVal(getVal($0,2,"\"","\""),1," "," ")
print "field7= "getVal(getVal($0,2,"\"","\""),2," "," ")
print "field8= "getVal($0,3,"\"","\"")
print "field9= "getVal($0,5,"\"","\"")
}' a

 

9 More Discussions You Might Find Interesting

1. Solaris

Apache localhost-access.log

The localhost-access.log has a size 3gb. What can apache2 break log on the parts 300mb, or the other issue, make log every week and index it with prifix current date(localhost-access_date.log)? Please help. (3 Replies)
Discussion started by: sotich82
3 Replies

2. Web Development

Apache log with long strings of Xs in GET request

Hi everybody, I was looking at my apache2 log and I found GET requests as such: Some ip - - "GET... (4 Replies)
Discussion started by: z1dane
4 Replies

3. Web Development

how to wait after apache log rotation

My solaris server utilize the freeware savelog program to rotate apache logs. One server has become very busy and even after doing a graceful restart it continues to log to the saved gzip log file. Has anyone been able to come up with a way or script to issue a "wait" type command so that the... (2 Replies)
Discussion started by: csross
2 Replies

4. Linux

Generating apache log reports

Hello all, I'm trying to find some tool on generating reports based on apache access_log files (of Common format). I found some of them (awstats, lire/logreport, weblog expert, apache logs viewer, etc..) but they generate some global and general report about the log file. Also some perl... (0 Replies)
Discussion started by: enux
0 Replies

5. Web Development

Apache/2.2.15 custom error log

Hello, I've updated my apache access log to include the x-forward-for IP instead of my client(loadbalancer) ip. However, i can't seem to find a way to do the same for the error logs. Can someone please assist. Thank you. -K (0 Replies)
Discussion started by: kmaq7621
0 Replies

6. Shell Programming and Scripting

Script that watches an apache log file

i'm trying to write a basic script that "watches" the apache access.log file and prints out lines that correspond to slow requests, Im checking for how microseconds it's taken. For the sake of testing, i'm using any number field. so far I have:- watch --interval=1 "tail access.log | cut -d '... (3 Replies)
Discussion started by: ssaini2014
3 Replies

7. UNIX for Advanced & Expert Users

Apache log rotate configuration

HI i was trying to configure logrotate for my apache server and it's not working properly. here is my lodrotate configuration /var/log/httpd/*log { daily missingok notifempty sharedscripts compress delaycompress postrotate /sbin/service httpd... (1 Reply)
Discussion started by: bentech4u
1 Replies

8. Red Hat

Apache log rotate configuration

HI i was trying to configure logrotate for my apache server and it's not working properly. Os: Red Hat 6 here is my lodrotate configuration /var/log/httpd/*log { daily missingok notifempty sharedscripts compress delaycompress postrotate ... (3 Replies)
Discussion started by: bentech4u
3 Replies

9. UNIX for Beginners Questions & Answers

Parse apache log file with three different time formats

Hi, I want to parse below file and Write a function to extract the logs between two given timestamp. Apache (Unix) Log Samples - MonitorWare The challenge here is there are three date and time format. First :- 07/Mar/2004:16:05:49 Second :- Sun Mar 7 16:02:00 2004 Third :- 29-Mar... (6 Replies)
Discussion started by: sahil_shine
6 Replies

LEARN ABOUT DEBIAN

apache::session::browseable

Apache::Session::Browseable(3pm)			User Contributed Perl Documentation			  Apache::Session::Browseable(3pm)

NAME

       Apache::Session::Browseable - Add index and search methods to Apache::Session

SYNOPSIS

	 use Apache::Session::Browseable::MySQL;

	 my $args = {
	      DataSource => 'dbi:mysql:sessions',
	      UserName	 => $db_user,
	      Password	 => $db_pass,
	      LockDataSource => 'dbi:mysql:sessions',
	      LockUserName   => $db_user,
	      LockPassword   => $db_pass,

	      # Choose your browseable fileds
	      Index	     => 'uid mail',
	 };

	 # Use it like Apache::Session
	 my %session;
	 tie %session, 'Apache::Session::Browseable::MySQL', $id, $args;
	 $session{uid} = 'me';
	 $session{mail} = 'me@me.com';
	 $session{unindexedField} = 'zz';
	 untie %session;

	 # Apache::Session::Browseable add some global class methods
	 #
	 # 1) search on a field (indexed or not)
	 my $hash = Apache::Session::Browseable::MySQL->searchOn( $args, 'uid', 'me' );
	 foreach my $id (keys %$hash) {
	   print $id . ":" . $hash->{$id}->{mail} . "
";
	 }

	 # 2) Parse all sessions
	 # a. get all sessions
	 my $hash = Apache::Session::Browseable::MySQL->get_key_from_all_sessions();

	 # b. get some fields from all sessions
	 my $hash = Apache::Session::Browseable::MySQL->get_key_from_all_sessions('uid', 'mail')

	 # c. execute something with datas from each session :
	 #    Example : get uid and mail if mail domain is
	 my $hash = Apache::Session::Browseable::MySQL->get_key_from_all_sessions(
		     sub {
			my ( $session, $id ) = @_;
			if ( $session->{mail} =~ /mydomain.com$/ ) {
			    return { $session->{uid}, $session->{mail} };
			}
		     }
	 );
	 foreach my $id (keys %$hash) {
	   print $id . ":" . $hash->{$id}->{uid} . "=>" . $hash->{$id}->{mail} . "
";
	 }

DESCRIPTION

       Apache::Session::browseable provides some class methods to manipulate all sessions and add the capability to index some fields to make
       research faster.

SEE ALSO

       Apache::Session

AUTHOR

       Xavier Guimard, <x.guimard@free.fr>

COPYRIGHT AND LICENSE

       Copyright (C) 2009 by Xavier Guimard

       This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.10.1 or,
       at your option, any later version of Perl 5 you may have available.

perl v5.14.2							    2009-10-31					  Apache::Session::Browseable(3pm)
All times are GMT -4. The time now is 10:26 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy