Help: Log data extract


Login or Register for Dates, Times and to Reply

 
Thread Tools Search this Thread
# 1  
Help: Log data extract

So I got this old webapp doing some strange format with its logging and I need to extract only specific parts.

Basically the format of the log goes:

Date:<space>timestamp
User:<space>userid
AppID:<space>applicationid
Duration:<space>time(ms)
<line>
<line>
Date:<space>timestamp
User:<space>userid
AppID:<space>applicationid
Duration:<space>time(ms)
<line>
<line>
Date:<space>timestamp
User:<space>userid
AppID:<space>applicationid
Duration: <space>time(ms)

..and so on.

I'd like to get the User, AppID and Duration values on days worth of log file and put them on a comma delimeted file in format:

User, AppID, Duration
User, AppID, Duration

I struggled with awk, but couldnt get it to work.. I appreciate any help, thanks
# 2  
Try and adapt the following script :
Code:
$ cat ken.sh
awk -v OFS=, '
function print_line() {
   if (user) {
      print user, appid, duration;
      user = "";
   }
}
$1 == "User:"     { print_line(); user = $2 }
$1 == "AppID:"    { appid = $2 }
$1 == "Duration:" { duration = $2 }
END               { print_line() }
' ken.log
$ cat ken.log
Date: 2007-10-01 13:00:00
User: userid1
AppID: applicationid1
Duration: 123
<line1>
<line2>
Date: 2007-10-01 13:00:05
User: userid2
AppID:applicationid2
Duration: 99
<line3>
<line4>
Date: 2007-19-01 13:01:00
User: userid1
AppID: applicationid3
Duration: 1087
<line4>
$ sh ken.sh
userid1,applicationid1,123
userid2,applicationid1,99
userid1,applicationid3,1087
$

Jean-Pierre.
Login or Register for Dates, Times and to Reply

Previous Thread | Next Thread
Thread Tools Search this Thread
Search this Thread:
Advanced Search

Test Your Knowledge in Computers #71
Difficulty: Easy
Linux, macOS, Android, iOS, Chrome OS, Orbis OS used on the PlayStation 4, whatever firmware is running on your router, all of these operating systems are generally referred to as a Unix-like operating system.
True or False?

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Extract data from log

I have logs in format ####<01-Mar-2015 03:48:18 o'clock GMT> <info> ####<01-Mar-2015 03:48:20 o'clock GMT> <info> ####<01-Mar-2015 03:48:30 o'clock GMT> <info> ####<01-Mar-2015 03:48:39 o'clock GMT> <info> I am looking out for a script which can extract data of last 15 minutes from the... (5 Replies)
Discussion started by: oberoi1403
5 Replies

2. Shell Programming and Scripting

Extract data from log file within specified time

So, we have a script, that is supposed to have a couple of functions like showing number of failed connections, recieved bytes per IP-address, and so on. We are supposed to be able to limit the number of results to either 0-24 hours or X days back from the last data in the log file. Everything... (3 Replies)
Discussion started by: Plumpen
3 Replies

3. UNIX for Dummies Questions & Answers

Extract date ranged data from log file

Hi, I am trying to extract lines of data within a log file on a Redhat 5 Linux system. eg I need all the lines with a particular username over the last 3 minutes. the log file may read like this, and I want a way to search all the lines extracting all the relevant lines over the last 3... (2 Replies)
Discussion started by: mantis100
2 Replies

4. Shell Programming and Scripting

Extract data from log file in specified range of time

I was searching for parsing a log file and found what I need in this link http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time But the most useful answer (posted by @Kent): # this variable you could customize, important is convert to seconds. # e.g... (2 Replies)
Discussion started by: kingk110
2 Replies

5. Shell Programming and Scripting

Data Extract from XML Log File

Please help me out to extract the Data from the XML Log files. So here is the data ERROR|2010-08-26 00:05:52,958|SERIAL_ID=128279996|ST=2010-08-2600:05:52|DEVICE=113.2.21.12:601|TYPE=TransactionLog... (9 Replies)
Discussion started by: raghunsi
9 Replies

6. Shell Programming and Scripting

How to extract log data based on date

Hi Gurus, I've been having some problem in extracting the log data based on the current date and month. As shown in the sample data below, how to extract the log info for Aug 11? Sample data: root pts/ta userpc Wed Aug 11 09:46 - 20:21 (10:35) root pts/ta userpc... (13 Replies)
Discussion started by: superHonda123
13 Replies

7. Shell Programming and Scripting

need a shell script to extract data from a log file.

If I have a log like : Mon Jul 19 05:07:34 2010; TCP; eth3; 52 bytes; from abc to def Mon Jul 19 05:07:35 2010; UDP; eth3; 46 bytes; from aaa to bbb Mon Jul 19 05:07:35 2010; TCP; eth3; 52 bytes; from def to ghi I will need an output like this : Time abc to def... (1 Reply)
Discussion started by: hitha87
1 Replies

8. Shell Programming and Scripting

Extract data from log file from or after the specific date

Hi , I am having a script which will start a process and appends the process related logs to a log file. The log file writes logs with every line starting with date in the format of: date +"%Y %b %d %H:%M:%S". So, in the script, before I start the process, I am storing the date as DATE=`date +"%Y... (5 Replies)
Discussion started by: chiru_h
5 Replies

9. Shell Programming and Scripting

shell-script which extract data from log file

give me a shell-script which extract data from log file on a server by giving date and time as input (for both start time and end time) and it will give the logs generated during the given time as output. (4 Replies)
Discussion started by: abhishek27
4 Replies

10. Shell Programming and Scripting

How to extract last week data from a log

HI, I have a log file (sync.log) in the following format. 05-14 05:34:56 INFO Hxxx Start: Id:xxx 05-14 05:45:32 INFO Hxxx End: Id:xxxx 05-19 11:23:21 INFO Hxxx Start: Id:xxx 05-19 11:34:45 INFO Hxxx End: Id:xxxx 05-20 02:09:47 INFO Hxxx Start: Id:xxx 05-20 02:36:03 INFO Hxxx End: Id:xxx... (5 Replies)
Discussion started by: praveenvi
5 Replies

Featured Tech Videos