Sponsored Content
Operating Systems Solaris how to grep or egrep pattern of apache access_log file Post 302313877 by otheus on Wednesday 6th of May 2009 07:34:10 PM
Old 05-06-2009
There's an easy solution, but the problem is that it's not exact. To understand what I mean, consider that Apache logs the requests as it completes them, but the date/time is when the request was made. It's therefore possible that extracting such a range will (a) include requests made (but not completed) before the start of that range and/or (b) exclude requests made within the time range but completed after other requests were made. So with that caveat, simply use awk:

Code:
awk '/02\/May\/2009:14:56:20/, /05\/May\/2009:18:46:06/' access_log

Oh, and since I deal with HTTP logs on a daily basis, one word of advice: change the custom log format to a tab-delimited one, like this:
Code:
LogFormat "%t\t%>s\t%b\t%r" tdf
CustomLog logs/access.tdf tdf

Obviously, you'll want more information in LogFormat -- this is a quick example. For more info, see Log Files - Apache HTTP Server. The main idea is to separate each field with tabs. From this, you can easily convert to the "Common-log" format if needed, but more importantly, you can more easily handle the logs directly with tools like awk.
Code:
awk -F\\t '$2 != 200'

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

egrep a certain pattern

hey guys this is my first post here, heard a lot about these forums. Iam urgently in need of a command which would help me accomplish the following , for example a file has these contents: 211 61 2007-06-26 13:47:32 211 61 2007-06-26 09:53:43 211 61 2007-06-26 15:25:14 211 61 2007-06-26... (5 Replies)
Discussion started by: trust123
5 Replies

2. Shell Programming and Scripting

HOW to egrep fo a pattern

Hi, I want to use egrep to match this expression in my file. The expression begins with the word SCHEDULE and ends with PFTDGNIN. In between these 2 words there can be anything. EX: Line1: SCHEDULE NWERRR#PFTDGNIN Line2: FOLLOWS NWD@AAS#PFTDGNIN So as a result of the egrep command... (1 Reply)
Discussion started by: eliewadi
1 Replies

3. Shell Programming and Scripting

Simple egrep pattern

I'm new to egrep. What pattern could I use to find all lines that match this pattern: <beginning of line><any amount of whitespace>sub<space>. I want it to return the entire line. (I'm trying to generate a list of all Perl sub definitions in a list of Perl modules.) Thanks for your help! (7 Replies)
Discussion started by: blondie53403
7 Replies

4. Shell Programming and Scripting

grep/egrep end of pattern

Hi I use arp to get the mac-addresses of my hosts. # arp -a | grep 192.168.0. e1000g0 192.168.0.1 255.255.255.255 o 00:00:00:00:00:01 e1000g0 192.168.0.11 255.255.255.255 o 00:00:00:00:00:02 e1000g0 192.168.0.2 255.255.255.255 ... (12 Replies)
Discussion started by: domi55
12 Replies

5. UNIX for Dummies Questions & Answers

Difference between grep, egrep & grep -i

Hi All, Please i need to know the difference between grep, egrep & grep -i when used to serach through a file. My platform is SunOS 5.9 & i'm using the korn shell. Regards, - divroro12 - (2 Replies)
Discussion started by: divroro12
2 Replies

6. Shell Programming and Scripting

How to egrep multiple pattern

Hi everyone i want to write a script to grep multiple pattern from all the file from a dir. for example I want to get all the record number from XML file who's last name is asd, smith, dfrt,gokul,and sinha. I tried egrep('sinha'|'gokul'|'asd') but it is not working also i tried saving... (2 Replies)
Discussion started by: revertback
2 Replies

7. Web Development

Include CFTOKEN and CFID in apache access_log

hi folks, how to write CFID and CFTOKEN cookie in apache logs ? can you give me a link or howtos in doing this. thanks in advance (0 Replies)
Discussion started by: linuxgeek
0 Replies

8. Linux

Finding IP info from access_log file

I found the /var/www/logs/access_log file (access log in order to find specific information about IP, And when users last logged in.) but in my fedora the access_log file is is in my /var/log/cups and it looks different from what it should be. Why is that? my goal is to get a list of IP... (4 Replies)
Discussion started by: bugenhagen_
4 Replies

9. Shell Programming and Scripting

sed and awk usage to grep a pattern 1 and with reference to this grep a pattern 2 and pattern 3

Hi , I have a file where i have modifed certain things compared to original file . The difference of the original file and modified file is as follows. # diff mir_lex.c.modified mir_lex.c.orig 3209c3209 < if(yy_current_buffer -> yy_is_our_buffer == 0) { --- >... (5 Replies)
Discussion started by: breezevinay
5 Replies

10. Shell Programming and Scripting

Egrep patterns in a file and limit number of matches to print for each pattern match

Hi I need to egrep patterns in a file and limit number of matches to print for each matched pattern. -m10 option is not working out in my sun solaris 5.10 Please guide me the options to achieve. if i do head -10 , i wont be getting all pattern match results as output since for a... (10 Replies)
Discussion started by: ananan
10 Replies
DBI::ProfileDumper::Apache(3)				User Contributed Perl Documentation			     DBI::ProfileDumper::Apache(3)

NAME
DBI::ProfileDumper::Apache - capture DBI profiling data from Apache/mod_perl SYNOPSIS
Add this line to your httpd.conf: PerlSetEnv DBI_PROFILE 2/DBI::ProfileDumper::Apache (If you're using mod_perl2, see "When using mod_perl2" for some additional notes.) Then restart your server. Access the code you wish to test using a web browser, then shutdown your server. This will create a set of dbi.prof.* files in your Apache log directory. Get a profiling report with dbiprof: dbiprof /path/to/your/apache/logs/dbi.prof.* When you're ready to perform another profiling run, delete the old files and start again. DESCRIPTION
This module interfaces DBI::ProfileDumper to Apache/mod_perl. Using this module you can collect profiling data from mod_perl applications. It works by creating a DBI::ProfileDumper data file for each Apache process. These files are created in your Apache log directory. You can then use the dbiprof utility to analyze the profile files. USAGE
LOADING THE MODULE The easiest way to use this module is just to set the DBI_PROFILE environment variable in your httpd.conf: PerlSetEnv DBI_PROFILE 2/DBI::ProfileDumper::Apache The DBI will look after loading and using the module when the first DBI handle is created. It's also possible to use this module by setting the Profile attribute of any DBI handle: $dbh->{Profile} = "2/DBI::ProfileDumper::Apache"; See DBI::ProfileDumper for more possibilities, and DBI::Profile for full details of the DBI's profiling mechanism. WRITING PROFILE DATA The profile data files will be written to your Apache log directory by default. The user that the httpd processes run as will need write access to the directory. So, for example, if you're running the child httpds as user 'nobody' and using chronolog to write to the logs directory, then you'll need to change the default. You can change the destination directory either by secifying a "Dir" value when creating the profile (like "File" in the DBI::ProfileDumper docs), or you can use the "DBI_PROFILE_APACHE_LOG_DIR" env var to change that. For example: PerlSetEnv DBI_PROFILE_APACHE_LOG_DIR /server_root/logs When using mod_perl2 Under mod_perl2 you'll need to either set the "DBI_PROFILE_APACHE_LOG_DIR" env var, or enable the mod_perl2 "GlobalRequest" option, like this: PerlOptions +GlobalRequest to the global config section you're about test with DBI::ProfileDumper::Apache. If you don't do one of those then you'll see messages in your error_log similar to: DBI::ProfileDumper::Apache on_destroy failed: Global $r object is not available. Set: PerlOptions +GlobalRequest in httpd.conf at ..../DBI/ProfileDumper/Apache.pm line 144 Naming the files The default file name is inherited from DBI::ProfileDumper via the filename() method, but DBI::ProfileDumper::Apache appends the parent pid and the current pid, separated by dots, to that name. Silencing the log By default a message is written to STDERR (i.e., the apache error_log file) when flush_to_disk() is called (either explicitly, or implicitly via DESTROY). That's usually very useful. If you don't want the log message you can silence it by setting the "Quiet" attribute true. PerlSetEnv DBI_PROFILE 2/DBI::ProfileDumper::Apache/Quiet:1 $dbh->{Profile} = "!Statement/DBI::ProfileDumper/Quiet:1"; $dbh->{Profile} = DBI::ProfileDumper->new( Path => [ '!Statement' ] Quiet => 1 ); GATHERING PROFILE DATA Once you have the module loaded, use your application as you normally would. Stop the webserver when your tests are complete. Profile data files will be produced when Apache exits and you'll see something like this in your error_log: DBI::ProfileDumper::Apache writing to /usr/local/apache/logs/dbi.prof.2604.2619 Now you can use dbiprof to examine the data: dbiprof /usr/local/apache/logs/dbi.prof.2604.* By passing dbiprof a list of all generated files, dbiprof will automatically merge them into one result set. You can also pass dbiprof sorting and querying options, see dbiprof for details. CLEANING UP Once you've made some code changes, you're ready to start again. First, delete the old profile data files: rm /usr/local/apache/logs/dbi.prof.* Then restart your server and get back to work. OTHER ISSUES
Memory usage DBI::Profile can use a lot of memory for very active applications because it collects profiling data in memory for each distinct query run. Calling "flush_to_disk()" will write the current data to disk and free the memory it's using. For example: $dbh->{Profile}->flush_to_disk() if $dbh->{Profile}; or, rather than flush every time, you could flush less often: $dbh->{Profile}->flush_to_disk() if $dbh->{Profile} and ++$i % 100; AUTHOR
Sam Tregar <sam@tregar.com> COPYRIGHT AND LICENSE
Copyright (C) 2002 Sam Tregar This program is free software; you can redistribute it and/or modify it under the same terms as Perl 5 itself. perl v5.12.1 2007-05-29 DBI::ProfileDumper::Apache(3)
All times are GMT -4. The time now is 04:49 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy