03-27-2009
Grep for lines in web logs
I want to find the unique url in a apache logs which got 404 error .
I can do something like
"cat apache.log|grep 404|awk '{print $2,$3}'|grep 404
this will give me say
/foo.html 404
/foo.html 404
/foo.html 404
/bar.html 404
/cat.html 404
However my output should only find unique results and should not give 3 instances of foo.html
It should look
/foo.html 404
/bar.html 404
/cat.html 404
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
We use an application that is dumping logs to a file on disk. However, this is dumping very verbosely and there is no method of turning down the logging level. We need to remove certain contents from these before they are commited to disk.
Has anybody got any ideas how I can do this... (3 Replies)
Discussion started by: harperonline
3 Replies
2. Shell Programming and Scripting
Hi,
I am trying to write a script which would go search and get the info from the logs based on yesterday timestamp and write yesterday logs in new file. The log file format is as follows:
""""""""""""""""""""""""""... (3 Replies)
Discussion started by: harish.parker
3 Replies
3. Shell Programming and Scripting
Hi,
could anyone help me out how to write a script, to grep the two timestamp from a particular file, so that it will list out all the logs between the particular timestamp
I have a pattern of log:
servicename operationname starttime endtime
eg.,
servicename1 operationname1 01:11:11... (1 Reply)
Discussion started by: jacktolearn
1 Replies
4. UNIX for Dummies Questions & Answers
Hello all. I am new to this forum and also very new to using grep - so please excuse me if this question is not in the correct forum and/or is not pertinent to this website.
I use greps like this:
gunzip -c L:\System1\SailLogger_20071019* L:\System2\SailLogger_20071019*... (14 Replies)
Discussion started by: Carl2013
14 Replies
5. Shell Programming and Scripting
Hi there,
I have a script that I've used to find errors in my Minecraft Server logs. But I'd like to refine that script to be more useful.
Here is the script:
grep -n "SEVERE" /minecraft/server.log | awk -F":" '{print $1-2 "," $1+10 "p"}' | xargs -t -i sed -n {} /minecraft/server.log >>... (15 Replies)
Discussion started by: nbsparks
15 Replies
6. Red Hat
Hello...
We keep having a problem with the httpd freezing and users getting time outs.
I am not the Sys Admin but since I am the 1st one in, I have to restart the httpd server.
Is there a list of items I should collect before I restart in order to help with trouble shooting?
I currently... (3 Replies)
Discussion started by: djlerman
3 Replies
7. Shell Programming and Scripting
Hi Guys,
I want to write a script which can grep the logs (server.log) from a file for Error String and output to a other file.
Problems:
How to know about the errors only between the current restart and not in previous as server.log has earlier restarts also?
thanks for the help! Much... (5 Replies)
Discussion started by: ankur328
5 Replies
8. Shell Programming and Scripting
Hello,
I need to write one script which should search particular pattern like ABCD in log file name hello.txt only in current date logs.
in current directory i have so many past date logs but grep should be applied on current date logs.
on daily basis current date logs are in number 30 and... (2 Replies)
Discussion started by: ajju
2 Replies
9. Shell Programming and Scripting
hey Guys,
Am a newbie to Linux and have a probable requirement that might come up pretty soon.
I need to understand, how log files and log location can be displayed on a web browser. So that the end user can read the log files and should be able to search for order numbers in the logs. If... (2 Replies)
Discussion started by: ashish_samarth
2 Replies
10. UNIX for Beginners Questions & Answers
Hi,
system date format
Thu Jun 13 12:55:18 EDT 2019
My log date format
09.148.192.60 - - "GET /akamai/sureroute-test-object.html HTTP/1.1" 404 231
can someone please help me, how to get last 5mins of logs please ? I need the command
Please wrap your samples/codes in CODE TAGS,... (3 Replies)
Discussion started by: scazed
3 Replies
LEARN ABOUT DEBIAN
test::pod::no404s
Test::Pod::No404s(3pm) User Contributed Perl Documentation Test::Pod::No404s(3pm)
NAME
Test::Pod::No404s - Checks POD for http 404 links
SYNOPSIS
#!/usr/bin/perl
use strict; use warnings;
use Test::More;
eval "use Test::Pod::No404s";
if ( $@ ) {
plan skip_all => 'Test::Pod::No404s required for testing POD';
} else {
all_pod_files_ok();
}
ABSTRACT
Using this test module will check your POD for any http 404 links.
DESCRIPTION
This module looks for any http(s) links in your POD and verifies that they will not return a 404. It uses LWP::UserAgent for the heavy
lifting, and simply lets you know if it failed to retrieve the document. More specifically, it uses $response->is_error as the "test."
Normally, you wouldn't want this test to be run during end-user installation because they might have no internet! It is HIGHLY recommended
that this be used only for module authors' RELEASE_TESTING phase. To do that, just modify the synopsis to add an env check :)
Methods
all_pod_files_ok( [ @files ] )
This function is what you will usually run. It automatically finds any POD in your distribution and runs checks on them.
Accepts an optional argument: an array of files to check. By default it checks all POD files it can find in the distribution. Every file it
finds is passed to the "pod_file_ok" function.
pod_file_ok( FILENAME, [ TESTNAME ] )
"pod_file_ok()" will okay the test if there is no http(s) links present in the POD or if all links are not an error. Furthermore, if the
POD was malformed as reported by Pod::Simple, the test will fail and not attempt to check the links.
When it fails, "pod_file_ok()" will show any failing links as diagnostics.
The optional second argument TESTNAME is the name of the test. If it is omitted, "pod_file_ok()" chooses a default test name "404 test for
FILENAME".
EXPORT
Automatically exports the two subs.
SEE ALSO
LWP::UserAgent
Pod::Simple
Test::Pod
SUPPORT
You can find documentation for this module with the perldoc command.
perldoc Test::Pod::No404s
Websites
o Search CPAN
<http://search.cpan.org/dist/Test-Pod-No404s>
o AnnoCPAN: Annotated CPAN documentation
<http://annocpan.org/dist/Test-Pod-No404s>
o CPAN Ratings
<http://cpanratings.perl.org/d/Test-Pod-No404s>
o CPAN Forum
<http://cpanforum.com/dist/Test-Pod-No404s>
o RT: CPAN's Request Tracker
<http://rt.cpan.org/NoAuth/Bugs.html?Dist=Test-Pod-No404s>
o CPANTS Kwalitee
<http://cpants.perl.org/dist/overview/Test-Pod-No404s>
o CPAN Testers Results
<http://cpantesters.org/distro/T/Test-Pod-No404s.html>
o CPAN Testers Matrix
<http://matrix.cpantesters.org/?dist=Test-Pod-No404s>
o Git Source Code Repository
<http://github.com/apocalypse/perl-test-pod-no404s>
Bugs
Please report any bugs or feature requests to "bug-test-pod-no404s at rt.cpan.org", or through the web interface at
<http://rt.cpan.org/NoAuth/ReportBug.html?Queue=Test-Pod-No404s>. I will be notified, and then you'll automatically be notified of
progress on your bug as I make changes.
AUTHOR
Apocalypse <apocal@cpan.org>
Thanks to the author of Test::Pod for the basic framework of this module!
Thanks to the POE guys for finding 404 links in their POD, and was the inspiration for this module.
COPYRIGHT AND LICENSE
Copyright 2010 by Apocalypse
This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.10.1 2011-03-05 Test::Pod::No404s(3pm)