07-27-2009
Generating formatted reports from log files
Given that I have a log file of the format:
DATE ID LOG_LEVEL | EVENT
2009-07-23T14:05:11Z T-4030097550 D | MessX
2009-07-23T14:10:44Z T-4030097550 D | MessY
2009-07-23T14:34:08Z T-7298651656 D | MessX
2009-07-23T14:41:00Z T-7298651656 D | MessY
2009-07-23T15:05:10Z T-4030097550 D | MessZ
etc
How do I produce a report that looks like
ID = T-4030097550
2009-07-23T14:05:11Z MessX
2009-07-23T14:10:44Z MessY
2009-07-23T15:05:10Z MessZ
ID = T-7298651656
2009-07-23T14:34:08Z MessX
2009-07-23T14:41:00Z MessY
etc
Other info:
The IDs are not pre-determined
The messages in the report are actually a subset of the log-entries and some messages may be missing for a given ID.
10 More Discussions You Might Find Interesting
1. Shell Programming and Scripting
Hi,
My aim is to generate a report using shell script. There are various formats fields coloumns etc.
I want to print in a single line (row) but in different coloumn as given below:
field1 field2 field3 field4
....... ....... ...... .......
The spacing... (1 Reply)
Discussion started by: jisha
1 Replies
2. Shell Programming and Scripting
hi i have data extracted in the following format ranging around 300000 to 800000 records in a text file , the format is of network data .
No. Time Source Destination Protocol
1 1998-06-05 17:20:23.569905 HP_61:aa:c9 HP_61:aa:c9 ... (1 Reply)
Discussion started by: renukaprasadb
1 Replies
3. Shell Programming and Scripting
I/P file name:- 20092008.txt
Check number of entries in i/p file by following command
ChkEnt -infl 20092008.txt -opfl 20092008_test.txt >count.txt
Dear Friends,
Please help me in automating following thing.
If output generated (count.txt) is having value more than 1000 i.e.... (8 Replies)
Discussion started by: anushree.a
8 Replies
4. Shell Programming and Scripting
Hi,
I have a question with sed/awk. When I handle some log files I want to search all reports with specified keyword. For example, in the log below.
abcd
efg
===start
abc
e
===end
xyz
===start
af
f
===end
nf
ga
===start
ab
===end (4 Replies)
Discussion started by: danielnpu
4 Replies
5. Red Hat
I installed sarg from sarg rpm and i am facing issue while generating sarg reports and getting this time different error below
sarg -l /var/log/squid/access.log
SARG: Records in file: 242332, reading: 0.00%
SARG: Records in file: 242332, reading: 2.06%
SARG: Records in file: 242332, reading:... (0 Replies)
Discussion started by: mail4vij
0 Replies
6. Shell Programming and Scripting
Good evening! Trying to make a shell script to parse log file and show only required information.
log file has 44 fields and alot of lines, each columns separated by ":".
log file is like:
first_1:3:4:5:6:1:3:4:5:something:notinterested
second_2:3:4:3:4:2
first_1:3:4:6:6:7:8
I am interested... (3 Replies)
Discussion started by: dummie55
3 Replies
7. Linux
Hello all,
I'm trying to find some tool on generating reports based on apache access_log files (of Common format).
I found some of them (awstats, lire/logreport, weblog expert, apache logs viewer, etc..) but they generate some global and general report about the log file.
Also some perl... (0 Replies)
Discussion started by: enux
0 Replies
8. Shell Programming and Scripting
Hi Experts,
I am having a requirement, where i need to generate a report of the execution time of all the processes. All the processes generate the log files in a log directory and I can get the execution time from the log files. like below is the log file.
/home/vikas/log >ls -l... (2 Replies)
Discussion started by: k_vikash
2 Replies
9. Shell Programming and Scripting
How to split the file and have suffix with formatted numbers
Tried the following code
awk '{filename="split."int((NR-1)/2)".txt"; print >> filename}' split.txt
Current Result
Expected Result (21 Replies)
Discussion started by: bobbygsk
21 Replies
10. UNIX for Beginners Questions & Answers
Team,
Hope you all are doing fine
I have one admin server which is being used dedicately to run cron jobs on hourly basis, fetching the details from Database which is in a different server.These cronjob are run on every hourly/5 minutes basis depending as per end user requirement.The script... (12 Replies)
Discussion started by: whizkidash
12 Replies
LEARN ABOUT DEBIAN
moosex::types::iso8601
MooseX::Types::ISO8601(3pm) User Contributed Perl Documentation MooseX::Types::ISO8601(3pm)
NAME
MooseX::Types::ISO8601 - ISO8601 date and duration string type constraints and coercions for Moose
SYNOPSIS
use MooseX::Types::ISO8601 qw/
ISO8601TimeDurationStr
/;
has duration => (
isa => ISO8601TimeDurationStr,
is => 'ro',
coerce => 1,
);
Class->new( duration => 60 ); # 60s => PT00H01M00S
Class->new( duration => DateTime::Duration->new(%args) )
DESCRIPTION
This module packages several TypeConstraints with coercions for working with ISO8601 date strings and the DateTime suite of objects.
DATE CONSTRAINTS
ISO8601DateStr
An ISO8601 date string. E.g. "2009-06-11"
ISO8601TimeStr
An ISO8601 time string. E.g. "12:06:34Z"
ISO8601DateTimeStr
An ISO8601 combined datetime string. E.g. "2009-06-11T12:06:34Z"
ISO8601DateTimeTZStr
An ISO8601 combined datetime string with a fully specified timezone. E.g. "2009-06-11T12:06:34+00:00"
COERCIONS
The date types will coerce from:
" Num "
The number is treated as a time in seconds since the unix epoch
" DateTime "
The duration represented as a DateTime object.
" Str "
Non-expanded date and time string representations.
e.g.:-
20120113 => 2012-01-13 170500Z => 17:05:00Z 20120113T170500Z => 2012-01-13T17:05:00Z
Representations of UTC time zone (only an offset of zero is supported)
e.g.:-
17:05:00+00:00 => 17:05:00Z 17:05:00+00 => 17:05:00Z 170500+0000 => 17:05:00Z
2012-01-13T17:05:00+00:00 => 2012-01-13T17:05:00Z 2012-01-13T17:05:00+00 => 2012-01-13T17:05:00Z 20120113T170500+0000 =>
2012-01-13T17:05:00Z
Also supports non-standards mixing of expanded and non-expanded representations
e.g.:-
2012-01-13T170500Z => 2012-01-13T17:05:00Z 20120113T17:05:00Z => 2012-01-13T17:05:00Z
DURATION CONSTRAINTS
ISO8601DateDurationStr
An ISO8601 date duration string. E.g. "P01Y01M01D"
ISO8601TimeDurationStr
An ISO8601 time duration string. E.g. "PT01H01M01S"
ISO8601DateTimeDurationStr
An ISO8601 comboined date and time duration string. E.g. "P01Y01M01DT01H01M01S"
COERCIONS
The duration types will coerce from:
" Num "
The number is treated as a time in seconds
" DateTime::Duration "
The duration represented as a DateTime::Duration object.
The duration types will coerce to:
" Duration "
A DateTime::Duration, i.e. the " Duration " constraint from MooseX::Types::DateTime.
SEE ALSO
o MooseX::Types::DateTime
o DateTime
o DateTime::Duration
o DateTime::Format::Duration
VERSION CONTROL
http://github.com/bobtfish/moosex-types-iso8601/tree/master
Patches are welcome.
SEE ALSO
o http://en.wikipedia.org/wiki/ISO_8601
o http://dotat.at/tmp/ISO_8601-2004_E.pdf
FEATURES
Fractional seconds
If provided, the number of seconds in time types is represented to microsecond accuracy. A full stop character is used as the decimal
seperator, which is allowed, but deprecated in preference to the comma character in ISO 8601:2004.
BUGS
Probably full of them, patches are very welcome.
Specifically missing features:
o No timezone support - all times are assumed UTC
o No week number type
o "Basic format", which lacks seperator characters, is not supported for reading or writing.
o Tests are rubbish.
AUTHOR
Tomas Doran (t0m) "<bobtfish@bobtfish.net>"
Dave Lambley "<davel@state51.co.uk>"
The development of this code was sponsored by my employer <http://www.state51.co.uk>.
Contributors
Aaron Moses
COPYRIGHT
Copyright (c) 2009 Tomas Doran. Some rights reserved.
This program is free software; you can redistribute
it and/or modify it under the same terms as Perl itself.
perl v5.14.2 2012-05-11 MooseX::Types::ISO8601(3pm)