Getting last section of data from logfile | Unix Linux Forums | Shell Programming and Scripting

  Go Back    


Shell Programming and Scripting Post questions about KSH, CSH, SH, BASH, PERL, PHP, SED, AWK and OTHER shell scripts and shell scripting languages here.

Getting last section of data from logfile

Shell Programming and Scripting


Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 12-04-2012
Jotne's Avatar
Jotne Jotne is offline
Registered User
 
Join Date: Dec 2010
Last Activity: 22 August 2014, 7:56 AM EDT
Posts: 1,040
Thanks: 62
Thanked 216 Times in 204 Posts
Getting last section of data from logfile

Hi, I have a log file from Munin like this:
Code:
2012/12/04 13:45:31 [INFO]: Munin-update finished (29.68 sec)
2012/12/04 13:50:01 Opened log file
2012/12/04 13:50:01 [INFO]: Starting munin-update
2012/12/04 13:50:01 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:50:01 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:50:11 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:21 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:31 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:50:33 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:50:33 [INFO]: Munin-update finished (31.71 sec)
2012/12/04 13:55:02 Opened log file
2012/12/04 13:55:02 [INFO]: Starting munin-update
2012/12/04 13:55:02 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:55:02 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_testing
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_testing
2012/12/04 13:55:12 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:22 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:32 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:37 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:55:37 [INFO]: Munin-update finished (35.48 sec)

How do I get only the last section from the log starting with last Opened log file so I get output like this:
Code:
2012/12/04 13:55:02 Opened log file
2012/12/04 13:55:02 [INFO]: Starting munin-update
2012/12/04 13:55:02 [ERROR] Error occured in under [] in the configuration.
2012/12/04 13:55:02 [Warning] Could not parse datafile /var/lib/munin/datafile: [ERROR] Error occured in under [] in the configuration.  Please refer to the log if details are missing here.
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_testing
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label pending_unstable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_stable
2012/12/04 13:55:02 [WARNING] Service apt_all on localhost.localdomain/127.0.0.1:4949 returned no data for label hold_testing
2012/12/04 13:55:12 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:22 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:32 [WARNING] Call to accept timed out.  Remaining workers: localdomain;localhost.localdomain
2012/12/04 13:55:37 [INFO] Reaping Munin::Master::UpdateWorker<localdomain;localhost.localdomain>.  Exit value/signal: 0/0
2012/12/04 13:55:37 [INFO]: Munin-update finished (35.48 sec)

Sponsored Links
    #2  
Old 12-04-2012
bartus11's Avatar
bartus11 bartus11 is offline Forum Staff  
Moderator
 
Join Date: Apr 2009
Last Activity: 2 September 2014, 5:27 PM EDT
Posts: 3,710
Thanks: 7
Thanked 1,142 Times in 1,113 Posts
Try:
Code:
perl -lp0e 's/.*\n([^\n]*Opened log file)/$1/s' file.log

Sponsored Links
    #3  
Old 12-04-2012
Vikram_Tanwar12 Vikram_Tanwar12 is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 21 November 2013, 3:32 AM EST
Posts: 168
Thanks: 13
Thanked 10 Times in 10 Posts
You can try this.


Code:
sed -n '/Opened log file/,$p' logfilename

It prints from the line matches the given pattern to end of file.
    #4  
Old 12-04-2012
Jotne's Avatar
Jotne Jotne is offline
Registered User
 
Join Date: Dec 2010
Last Activity: 22 August 2014, 7:56 AM EDT
Posts: 1,040
Thanks: 62
Thanked 216 Times in 204 Posts
I would like to have all data from last found Opened log file , not from first found
Prefer sed/awk soultion. My knowledge about perl is missing.
Sponsored Links
    #5  
Old 12-04-2012
bartus11's Avatar
bartus11 bartus11 is offline Forum Staff  
Moderator
 
Join Date: Apr 2009
Last Activity: 2 September 2014, 5:27 PM EDT
Posts: 3,710
Thanks: 7
Thanked 1,142 Times in 1,113 Posts
Quote:
Originally Posted by Jotne View Post
I would like to have all data from last found Opened log file , not from first found
Prefer sed/awk soultion. My knowledge about perl is missing.
Did you try my code?
Sponsored Links
    #6  
Old 12-04-2012
Jotne's Avatar
Jotne Jotne is offline
Registered User
 
Join Date: Dec 2010
Last Activity: 22 August 2014, 7:56 AM EDT
Posts: 1,040
Thanks: 62
Thanked 216 Times in 204 Posts
Yes, your solution works
But prefer not to use perl



Code:
awk '/Opened log file/ {i=NR} END {for (x=i;x>i;i++) {print $NR}}' munin-update.log

This does not work, but it should be some like it.
Find last hits, by setting i to NR
Then print all record form i and out

Last edited by Jotne; 12-04-2012 at 09:10 AM..
Sponsored Links
    #7  
Old 12-04-2012
Vikram_Tanwar12 Vikram_Tanwar12 is offline
Registered User
 
Join Date: Nov 2012
Last Activity: 21 November 2013, 3:32 AM EST
Posts: 168
Thanks: 13
Thanked 10 Times in 10 Posts
Then you can open the log file and type in the end


Code:
set nu

and then you just need to see the line number of the last Opened log file suppose its line number is 4 and the last line number is 50 then you just need to type on command line.


Code:
sed -n '4,50'p logfilename

It will give result as per your need.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
take a section of a data with conditions saeed.soltani Shell Programming and Scripting 3 08-26-2012 01:18 AM
extracting data froma logfile DOkuwa Shell Programming and Scripting 3 01-05-2011 07:08 PM
Extract section of file based on word in section jelloir Shell Programming and Scripting 2 09-20-2010 02:16 AM
Search and Remove No data Section petersf Shell Programming and Scripting 2 02-08-2010 02:34 PM
get last section from large logfile kburrows Shell Programming and Scripting 9 05-23-2004 07:06 PM



All times are GMT -4. The time now is 10:05 PM.