Sponsored Content
Top Forums Shell Programming and Scripting Generating formatted reports from log files Post 302338045 by daccad on Monday 27th of July 2009 03:33:12 AM
Old 07-27-2009
Generating formatted reports from log files

Given that I have a log file of the format:

DATE ID LOG_LEVEL | EVENT
2009-07-23T14:05:11Z T-4030097550 D | MessX
2009-07-23T14:10:44Z T-4030097550 D | MessY
2009-07-23T14:34:08Z T-7298651656 D | MessX
2009-07-23T14:41:00Z T-7298651656 D | MessY
2009-07-23T15:05:10Z T-4030097550 D | MessZ
etc

How do I produce a report that looks like

ID = T-4030097550
2009-07-23T14:05:11Z MessX
2009-07-23T14:10:44Z MessY
2009-07-23T15:05:10Z MessZ
ID = T-7298651656
2009-07-23T14:34:08Z MessX
2009-07-23T14:41:00Z MessY
etc

Other info:
The IDs are not pre-determined
The messages in the report are actually a subset of the log-entries and some messages may be missing for a given ID.
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Generating a report -Formatted printing -Urgent

Hi, My aim is to generate a report using shell script. There are various formats fields coloumns etc. I want to print in a single line (row) but in different coloumn as given below: field1 field2 field3 field4 ....... ....... ...... ....... The spacing... (1 Reply)
Discussion started by: jisha
1 Replies

2. Shell Programming and Scripting

generating reports based on time field of network data

hi i have data extracted in the following format ranging around 300000 to 800000 records in a text file , the format is of network data . No. Time Source Destination Protocol 1 1998-06-05 17:20:23.569905 HP_61:aa:c9 HP_61:aa:c9 ... (1 Reply)
Discussion started by: renukaprasadb
1 Replies

3. Shell Programming and Scripting

Generating files.

I/P file name:- 20092008.txt Check number of entries in i/p file by following command ChkEnt -infl 20092008.txt -opfl 20092008_test.txt >count.txt Dear Friends, Please help me in automating following thing. If output generated (count.txt) is having value more than 1000 i.e.... (8 Replies)
Discussion started by: anushree.a
8 Replies

4. Shell Programming and Scripting

how to search reports with specified keyword in log

Hi, I have a question with sed/awk. When I handle some log files I want to search all reports with specified keyword. For example, in the log below. abcd efg ===start abc e ===end xyz ===start af f ===end nf ga ===start ab ===end (4 Replies)
Discussion started by: danielnpu
4 Replies

5. Red Hat

sarg issue while generating reports for squid

I installed sarg from sarg rpm and i am facing issue while generating sarg reports and getting this time different error below sarg -l /var/log/squid/access.log SARG: Records in file: 242332, reading: 0.00% SARG: Records in file: 242332, reading: 2.06% SARG: Records in file: 242332, reading:... (0 Replies)
Discussion started by: mail4vij
0 Replies

6. Shell Programming and Scripting

Extract information from Log file formatted

Good evening! Trying to make a shell script to parse log file and show only required information. log file has 44 fields and alot of lines, each columns separated by ":". log file is like: first_1:3:4:5:6:1:3:4:5:something:notinterested second_2:3:4:3:4:2 first_1:3:4:6:6:7:8 I am interested... (3 Replies)
Discussion started by: dummie55
3 Replies

7. Linux

Generating apache log reports

Hello all, I'm trying to find some tool on generating reports based on apache access_log files (of Common format). I found some of them (awstats, lire/logreport, weblog expert, apache logs viewer, etc..) but they generate some global and general report about the log file. Also some perl... (0 Replies)
Discussion started by: enux
0 Replies

8. Shell Programming and Scripting

Generate a Execution time reports using log files.

Hi Experts, I am having a requirement, where i need to generate a report of the execution time of all the processes. All the processes generate the log files in a log directory and I can get the execution time from the log files. like below is the log file. /home/vikas/log >ls -l... (2 Replies)
Discussion started by: k_vikash
2 Replies

9. Shell Programming and Scripting

Split files with formatted numbers

How to split the file and have suffix with formatted numbers Tried the following code awk '{filename="split."int((NR-1)/2)".txt"; print >> filename}' split.txt Current Result Expected Result (21 Replies)
Discussion started by: bobbygsk
21 Replies

10. UNIX for Beginners Questions & Answers

Cron job scheduled is running once, but reports are generating twice

Team, Hope you all are doing fine I have one admin server which is being used dedicately to run cron jobs on hourly basis, fetching the details from Database which is in a different server.These cronjob are run on every hourly/5 minutes basis depending as per end user requirement.The script... (12 Replies)
Discussion started by: whizkidash
12 Replies
W3C_EXTENDED2DLF.IN(1)					  LogReport's Lire Documentation				    W3C_EXTENDED2DLF.IN(1)

NAME
w3c_extended2dlf - convert W3C Extended Log file as used by MS IIS to DLF SYNOPSIS
w3c_extended2dlf file DESCRIPTION
w3c_extended2dlf converts web server log files in the W3C Extended Log Format to the www DLF. This log format is defined at http://www.w3.org/TR/WD-logfile.html It is used by IIS 4.0 and IIS 5.0. This is a customizable format which contains an header specifying which information is present in the log. Some documenation on these log file formats is available in the section on "Analyzing Log Files (IIS 6.0)" in the "IIS 6.0 Documentation" at the "Microsoft Windows Server 2003 TechCenter" at http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/ Library/IIS/610b7d2c-90d6-4e40-be79-aaf88a283f03.mspx?mfr=true . The newer IIS 7.0 product is documented at http://www.iis.net/. To have the maximum information in you reports, we suggests that you log the following fields : date, time, c-ip, c-dns, cs-uri, cs-method, sc-bytes, sc-status, cs(User-Agent), cs(Referer) and cs-username We also support the cs-uri-stem field. Other fields will be ignored. LIMITATIONS
The converter doesn't handle aggregation (record with count field) and will refuse to process those logs. Also it doesn't support changing the fields in the middle of the log file. It will ignore records that don't have the same schema than the first one defined. EXAMPLES
To process a log as produced in the W3C Extended Log Format: $ w3c_extended2dlf < extended.log w3c_extended2dlf will be rarely used on its own, but is more likely called by lr_log2report: $ lr_log2report w3c_extended < /var/log/httpd/extended.log AUTHORS
Francis J. Lacoste <flacoste@logreport.org> VERSION
$Id: w3c_extended2dlf.in,v 1.17 2009/03/15 08:10:55 vanbaal Exp $ COPYRIGHT
Copyright (C) 2001, 2002 Stichting LogReport Foundation LogReport@LogReport.org This program is part of Lire. Lire is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program (see COPYING); if not, check with http://www.gnu.org/copyleft/gpl.html. Lire 2.1.1 2009-03-15 W3C_EXTENDED2DLF.IN(1)
All times are GMT -4. The time now is 08:54 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy