07-27-2009
Generating formatted reports from log files
Given that I have a log file of the format:
DATE ID LOG_LEVEL | EVENT
2009-07-23T14:05:11Z T-4030097550 D | MessX
2009-07-23T14:10:44Z T-4030097550 D | MessY
2009-07-23T14:34:08Z T-7298651656 D | MessX
2009-07-23T14:41:00Z T-7298651656 D | MessY
2009-07-23T15:05:10Z T-4030097550 D | MessZ
etc
How do I produce a report that looks like
ID = T-4030097550
2009-07-23T14:05:11Z MessX
2009-07-23T14:10:44Z MessY
2009-07-23T15:05:10Z MessZ
ID = T-7298651656
2009-07-23T14:34:08Z MessX
2009-07-23T14:41:00Z MessY
etc
Other info:
The IDs are not pre-determined
The messages in the report are actually a subset of the log-entries and some messages may be missing for a given ID.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
My aim is to generate a report using shell script. There are various formats fields coloumns etc.
I want to print in a single line (row) but in different coloumn as given below:
field1 field2 field3 field4
....... ....... ...... .......
The spacing... (1 Reply)
Discussion started by: jisha
1 Replies
2. Shell Programming and Scripting
hi i have data extracted in the following format ranging around 300000 to 800000 records in a text file , the format is of network data .
No. Time Source Destination Protocol
1 1998-06-05 17:20:23.569905 HP_61:aa:c9 HP_61:aa:c9 ... (1 Reply)
Discussion started by: renukaprasadb
1 Replies
3. Shell Programming and Scripting
I/P file name:- 20092008.txt
Check number of entries in i/p file by following command
ChkEnt -infl 20092008.txt -opfl 20092008_test.txt >count.txt
Dear Friends,
Please help me in automating following thing.
If output generated (count.txt) is having value more than 1000 i.e.... (8 Replies)
Discussion started by: anushree.a
8 Replies
4. Shell Programming and Scripting
Hi,
I have a question with sed/awk. When I handle some log files I want to search all reports with specified keyword. For example, in the log below.
abcd
efg
===start
abc
e
===end
xyz
===start
af
f
===end
nf
ga
===start
ab
===end (4 Replies)
Discussion started by: danielnpu
4 Replies
5. Red Hat
I installed sarg from sarg rpm and i am facing issue while generating sarg reports and getting this time different error below
sarg -l /var/log/squid/access.log
SARG: Records in file: 242332, reading: 0.00%
SARG: Records in file: 242332, reading: 2.06%
SARG: Records in file: 242332, reading:... (0 Replies)
Discussion started by: mail4vij
0 Replies
6. Shell Programming and Scripting
Good evening! Trying to make a shell script to parse log file and show only required information.
log file has 44 fields and alot of lines, each columns separated by ":".
log file is like:
first_1:3:4:5:6:1:3:4:5:something:notinterested
second_2:3:4:3:4:2
first_1:3:4:6:6:7:8
I am interested... (3 Replies)
Discussion started by: dummie55
3 Replies
7. Linux
Hello all,
I'm trying to find some tool on generating reports based on apache access_log files (of Common format).
I found some of them (awstats, lire/logreport, weblog expert, apache logs viewer, etc..) but they generate some global and general report about the log file.
Also some perl... (0 Replies)
Discussion started by: enux
0 Replies
8. Shell Programming and Scripting
Hi Experts,
I am having a requirement, where i need to generate a report of the execution time of all the processes. All the processes generate the log files in a log directory and I can get the execution time from the log files. like below is the log file.
/home/vikas/log >ls -l... (2 Replies)
Discussion started by: k_vikash
2 Replies
9. Shell Programming and Scripting
How to split the file and have suffix with formatted numbers
Tried the following code
awk '{filename="split."int((NR-1)/2)".txt"; print >> filename}' split.txt
Current Result
Expected Result (21 Replies)
Discussion started by: bobbygsk
21 Replies
10. UNIX for Beginners Questions & Answers
Team,
Hope you all are doing fine
I have one admin server which is being used dedicately to run cron jobs on hourly basis, fetching the details from Database which is in a different server.These cronjob are run on every hourly/5 minutes basis depending as per end user requirement.The script... (12 Replies)
Discussion started by: whizkidash
12 Replies
LEARN ABOUT DEBIAN
test::log::dispatch
Test::Log::Dispatch(3pm) User Contributed Perl Documentation Test::Log::Dispatch(3pm)
NAME
Test::Log::Dispatch -- Test what you are logging
SYNOPSIS
use Test::More;
use Test::Log::Dispatch;
my $log = Test::Log::Dispatch->new();
# ...
# call something that logs to $log
# ...
# now test to make sure you logged the right things
$log->contains_ok(qr/good log message/, "good message was logged");
$log->does_not_contain_ok(qr/unexpected log message/, "unexpected message was not logged");
$log->empty_ok("no more logs");
# or
my $msgs = $log->msgs;
cmp_deeply($msgs, ['msg1', 'msg2', 'msg3']);
DESCRIPTION
"Test::Log::Dispatch" is a "Log::Dispatch" object that keeps track of everything logged to it in memory, and provides convenient tests
against what has been logged.
CONSTRUCTOR
The constructor returns a "Test::Log::Dispatch" object, which inherits from "Log::Dispatch" and contains a single "Log::Dispatch::Array"
output at 'debug' level.
The constructor requires no parameters. Any parameters will be forwarded to the "Log::Dispatch::Array" constructor. For example, you can
pass a min_level to override the default 'debug'.
METHODS
The test_name is optional in the *_ok methods; a reasonable default will be provided.
contains_ok ($regex[, $test_name])
Tests that a message in the log buffer matches $regex. On success, the message is removed from the log buffer (but any other matches
are left untouched).
does_not_contain_ok ($regex[, $test_name])
Tests that no message in the log buffer matches $regex.
empty_ok ([$test_name])
Tests that there is no log buffer left. On failure, the log buffer is cleared to limit further cascading failures.
contains_only_ok ($regex[, $test_name])
Tests that there is a single message in the log buffer and it matches $regex. On success, the message is removed.
clear ()
Clears the log buffer.
msgs ()
Returns the current contents of the log buffer as an array reference, where each element is a hash containing a message and level key.
TO DO
o Allow testing of log levels.
SEE ALSO
Log::Dispatch, Test::Log4perl
AUTHOR
Jonathan Swartz
COPYRIGHT & LICENSE
Copyright (C) 2009 Jonathan Swartz, all rights reserved.
This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.
perl v5.10.1 2009-11-04 Test::Log::Dispatch(3pm)