Sponsored Content
Top Forums Shell Programming and Scripting How to search backwards in a log file by timestamp of entries? Post 302356691 by varontron on Sunday 27th of September 2009 12:23:49 AM
Old 09-27-2009
use File::ReadBackwards

see how-to: reading a file backwards | Perl HowTo

Code:
#!/usr/bin/perl
use File::ReadBackwards;
 
$fh = File::ReadBackwards->new('access.log') or die "can't read file: $!\n";
 
while ( defined($line = $fh->readline) )
{
  if ($line =~ /regex to capture time/)
  {
      #get max time on first iteration
      #check time against max time
      #if within range, add to array
      #otherwise, exit the loop
  }
}
foreach my $line (reverse @lines)
{
   #process each line as needed
}

 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Spooling a log file with timestamp

Hi From shell script i am invoking sqlplus to connect to oracle database and then i spool a csv file as with output. What i want to do is to change the file name with timestamp on it so after spooling finish shell script change file name with time stamp. can someone help me to do that . Thanks... (2 Replies)
Discussion started by: ukadmin
2 Replies

2. UNIX for Advanced & Expert Users

Copy lines from a log file based on timestamp

how to copy lines from a log file based on timestamp. INFO (RbrProcessFlifoEventSessionEJB.java:processFlight:274) - E_20080521_110754_967: rbrAciInfoObjects listing complete! INFO (RbrPnrProcessEventSessionEJB.java:processFlight:197) - Event Seq: 1647575217; Carrier: UA; Flt#: 0106; Origin:... (1 Reply)
Discussion started by: ranjiadmin
1 Replies

3. Shell Programming and Scripting

Search backwards

Hi, I have a variable , lets say a=/disk1/net/first.ksh i need to grep "first.ksh" everytime "a" gets changed dynamically and i do not know how many '"/" are there in my variable. Can somebody help me out. (9 Replies)
Discussion started by: giri_luck
9 Replies

4. Shell Programming and Scripting

search backwards relative to a string

Hi, I have to search for first occurenceof string str1 in a file(>5GB). Now, after I have that , I have to search backwards from that offset till I get another string str2. I should also be able to get the new string str2's offset. Similarly, I look for last occurence of str1 and then... (1 Reply)
Discussion started by: finder255
1 Replies

5. Shell Programming and Scripting

concatenate log file lines up to timestamp

Hi, Using sed awk or perl I am trying to do something similar to https://www.unix.com/shell-programming-scripting/105887-sed-awk-concatenate-lines-until-blank-line-2.html but my requirement is slightly different. What I am trying to accomplish is to reformat a logfile such that all lines... (4 Replies)
Discussion started by: AlanC
4 Replies

6. Shell Programming and Scripting

search for string and replace backwards

I'm new to Unix scripting and I'm not sure if this can be done. Example: search (grep) in a file for 'Control ID' and then replace with 4 blanks 7 bytes before 'Control ID. input "xxxxxx1234xxxxxxxControl IDxxxxxx" output: "xxxxxx xxxxxxxControl IDxxxxxx" thanks! (7 Replies)
Discussion started by: jbt828
7 Replies

7. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

8. Programming

How to search a file based on a time stamp backwards 10 seconds

Hi all, I'm after some help with this small issue which i'm struggling to work out a fix for. I have a file that contains records that all have a time stamp for each individual record, i need to search the file for a specific time stamp and then search back 10 seconds to see if the number... (2 Replies)
Discussion started by: sp3arsy
2 Replies

9. Shell Programming and Scripting

Search backwards to certain string

Hi, I'm using the following to do a backwards search of a file for a string sed s/^M//g FILE | nawk 'c-->0;$0~s{if(b)for(c=b+1;c>1;c--)print r;print;c=a}b{r=$0}' b=10 a=0 s="9005"|grep "policy "|sort -u |awk '{print $4}'|cut -c2-10 My issue is that because I'm looking back 10 lines it's... (11 Replies)
Discussion started by: SaltyDog
11 Replies

10. Shell Programming and Scripting

Search for latest Timestamp in file

Hi, I have a file which generate Timestamp in this format :- 20121012162354 20121114191610 20121210232808 20121216220002 20130106220002 20130127220001 20130203220001 20121012162354 20121114191610 20121210232808 20121216220002 20130106220002 20130127220001 20130203220001 (2 Replies)
Discussion started by: netdbaind
2 Replies
DBD::File::HowTo(3)					User Contributed Perl Documentation				       DBD::File::HowTo(3)

NAME
DBD::File::HowTo - Guide to create DBD::File based driver SYNOPSIS
perldoc DBD::File::HowTo perldoc DBI perldoc DBI::DBD perldoc DBD::File::Developers perldoc DBI::DBD::SqlEngine::Developers perldoc DBI::DBD::SqlEngine perldoc SQL::Eval perldoc DBI::DBD::SqlEngine::HowTo perldoc SQL::Statement::Embed perldoc DBD::File perldoc DBD::File::HowTo perldoc DBD::File::Developers DESCRIPTION
This document provides a step-by-step guide, how to create a new "DBD::File" based DBD. It expects that you carefully read the DBI documentation and that you're familiar with DBI::DBD and had read and understood DBD::ExampleP. This document addresses experienced developers who are really sure that they need to invest time when writing a new DBI Driver. Writing a DBI Driver is neither a weekend project nor an easy job for hobby coders after work. Expect one or two man-month of time for the first start. Those who are still reading, should be able to sing the rules of "CREATING A NEW DRIVER" in DBI::DBD. Of course, DBD::File is a DBI::DBD::SqlEngine and you surely read DBI::DBD::SqlEngine::HowTo before continuing here. CREATING DRIVER CLASSES
Do you have an entry in DBI's DBD registry? For this guide, a prefix of "foo_" is assumed. Sample Skeleton package DBD::Foo; use strict; use warnings; use vars qw(@ISA $VERSION); use base qw(DBD::File); use DBI (); $VERSION = "0.001"; package DBD::Foo::dr; use vars qw(@ISA $imp_data_size); @ISA = qw(DBD::File::dr); $imp_data_size = 0; package DBD::Foo::db; use vars qw(@ISA $imp_data_size); @ISA = qw(DBD::File::db); $imp_data_size = 0; package DBD::Foo::st; use vars qw(@ISA $imp_data_size); @ISA = qw(DBD::File::st); $imp_data_size = 0; package DBD::Foo::Statement; use vars qw(@ISA); @ISA = qw(DBD::File::Statement); package DBD::Foo::Table; use vars qw(@ISA); @ISA = qw(DBD::File::Table); 1; Tiny, eh? And all you have now is a DBD named foo which will is able to deal with temporary tables, as long as you use SQL::Statement. In DBI::SQL::Nano environments, this DBD can do nothing. Start over Based on DBI::DBD::SqlEngine::HowTo, we're now having a driver which could do basic things. Of course, it should now derive from DBD::File instead of DBI::DBD::SqlEngine, shouldn't it? DBD::File extends DBI::DBD::SqlEngine to deal with any kind of files. In principle, the only extensions required are to the table class: package DBD::Foo::Table; sub bootstrap_table_meta { my ( $self, $dbh, $meta, $table ) = @_; # initialize all $meta attributes which might be relevant for # file2table return $self->SUPER::bootstrap_table_meta($dbh, $meta, $table); } sub init_table_meta { my ( $self, $dbh, $meta, $table ) = @_; # called after $meta contains the results from file2table # initialize all missing $meta attributes $self->SUPER::init_table_meta( $dbh, $meta, $table ); } In case "DBD::File::Table::open_file" doesn't open the files as the driver needs that, override it! sub open_file { my ( $self, $meta, $attrs, $flags ) = @_; # ensure that $meta->{f_dontopen} is set $self->SUPER::open_file( $meta, $attrs, $flags ); # now do what ever needs to be done } Combined with the methods implemented using the SQL::Statement::Embed guide, the table is full working and you could try a start over. User comfort "DBD::File" since 0.39 consolidates all persistent meta data of a table into a single structure stored in "$dbh->{f_meta}". With "DBD::File" version 0.41 and "DBI::DBD::SqlEngine" version 0.05, this consolidation moves to DBI::DBD::SqlEngine. It's still the "$dbh->{$drv_prefix . "_meta"}" attribute which cares, so what you learned at this place before, is still valid. sub init_valid_attributes { my $dbh = $_[0]; $dbh->SUPER::init_valid_attributes (); $dbh->{foo_valid_attrs} = { ... }; $dbh->{foo_readonly_attrs} = { ... }; $dbh->{foo_meta} = "foo_tables"; return $dbh; } See updates at "User comfort" in DBI::DBD::SqlEngine::HowTo. Testing Now you should have your own DBD::File based driver. Was easy, wasn't it? But does it work well? Prove it by writing tests and remember to use dbd_edit_mm_attribs from DBI::DBD to ensure testing even rare cases. AUTHOR
This guide is written by Jens Rehsack. DBD::File is written by Jochen Wiedmann and Jeff Zucker. The module DBD::File is currently maintained by H.Merijn Brand < h.m.brand at xs4all.nl > and Jens Rehsack < rehsack at googlemail.com > COPYRIGHT AND LICENSE
Copyright (C) 2010 by H.Merijn Brand & Jens Rehsack All rights reserved. You may freely distribute and/or modify this module under the terms of either the GNU General Public License (GPL) or the Artistic License, as specified in the Perl README file. perl v5.18.2 2013-04-04 DBD::File::HowTo(3)
All times are GMT -4. The time now is 08:55 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy