Sponsored Content
Top Forums UNIX for Beginners Questions & Answers Filter records from a log file based on timestamp Post 303043479 by mukulverma2408 on Wednesday 29th of January 2020 11:56:01 AM
Old 01-29-2020
Dear Experts,
This is what I've tried by converting into epoch time and it's giving me the desired result however it seems to me like this is not the standard way of doing this, do you mind taking a look and improve this solution pls.
Code:
dt1=`date -d 'Jan 19 21:15:00' +%s`
#echo $dt1
while read line
do
	tmstmp=$(echo $line | cut -c '1-15' )
	#echo $tmstmp
	dt2=`date -d "$tmstmp" +%s`
	#echo  $dt2
	echo "Ref time in epoch is $dt1"
	echo "Timestamp from file is $tmstmp"
	echo "Timestamp from file in epoch is $dt2"
	if [[ $dt2 -le $dt1 ]]
	then
		echo "This event occured before 21:15, inserting into new log file"
		echo $line >> filter.txt
	else
		continue 
		
	fi		
	
done < auth.log

 

10 More Discussions You Might Find Interesting

1. UNIX for Advanced & Expert Users

Copy lines from a log file based on timestamp

how to copy lines from a log file based on timestamp. INFO (RbrProcessFlifoEventSessionEJB.java:processFlight:274) - E_20080521_110754_967: rbrAciInfoObjects listing complete! INFO (RbrPnrProcessEventSessionEJB.java:processFlight:197) - Event Seq: 1647575217; Carrier: UA; Flt#: 0106; Origin:... (1 Reply)
Discussion started by: ranjiadmin
1 Replies

2. Shell Programming and Scripting

List all log records logged after $timestamp ?

I am trying to find a way to list every records inside a file (usually a log file) that are present after a record mathing/greater-then a timestamp supplied by another script. The timestamp can be anywhere inside the record and it is usually in the standard `date` format (will not look for other... (5 Replies)
Discussion started by: Browser_ice
5 Replies

3. Shell Programming and Scripting

filter out all the records which are having space in the 8th filed of my file

I have a file which is having fileds separtaed by delimiter. Ex: C;4498;qwa;cghy;;;;40;;222122 C;4498;sample;city;;;;34 2;;222123 C;4498;qwe;xcbv;;;;34-2;;222124 C;4498;jj;sffz;;;;41;;222120 C;4498;eert;qwq;;;;34 A;;222125 C;4498;jj;szxzzd;;;;34;;222127 out of these records I... (3 Replies)
Discussion started by: indusri
3 Replies

4. Shell Programming and Scripting

Filter records in a file using AWK

I want to filter records in one of my file using AWK command (or anyother command). I am using the below code awk -F@ '$1=="0003"&&"$2==20100402" print {$0}' $INPUT > $OUTPUT I want to pass the 0003 and 20100402 values through a variable. How can I do this? Any help is much... (1 Reply)
Discussion started by: gpaulose
1 Replies

5. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

6. Shell Programming and Scripting

Need to get all the records from a log file greater than timestamp supplied.

I have a log file which has records with hung thread information/error I need to find out hung thread from log file greater than timestamp supplied. 00000026 ThreadMonitor W WSVR0605W: Thread "WebContainer : 1" (00000027) has been active for 701879 milliseconds and may be hung. There is/are... (6 Replies)
Discussion started by: megh
6 Replies

7. Shell Programming and Scripting

Filter records based on 2nd file

Hello, I want to filter records of a file if they fall in range associated with a second file. First the chr number (2nd col of 1st file and 1st col of 2nd file) needs to be matched. Then if the 3rd col of the first file falls within any of the ranges specified by the 2nd and 3rd cols , then... (4 Replies)
Discussion started by: ritakadm
4 Replies

8. UNIX for Dummies Questions & Answers

Filter records in a huge text file from a filter text file

Hi Folks, I have a text file with lots of rows with duplicates in the first column, i want to filter out records based on filter columns in a different filter text file. bash scripting is what i need. Data.txt Name OrderID Quantity Sam 123 300 Jay 342 498 Kev 78 2500 Sam 420 50 Vic 10... (3 Replies)
Discussion started by: tech_frk
3 Replies

9. Shell Programming and Scripting

Shell script to filter records in a zip file that contains matching columns from another file

Not sure if this is the correct forum for this question. I have two files. file1.zip, file2 Input: file1.zip col1, col2 , col3 a , b , 0:0:0:0:0:c436:9346:d40b x, y, 0:0:0:0:0:880:39f9:c9a7 m, n , 0:0:0:0:0:80c7:9161:fe00 file2.txt col1 c4:36:93:46:d4:0b... (1 Reply)
Discussion started by: anil.v
1 Replies

10. Shell Programming and Scripting

Awk/sed/cut to filter out records from a file based on criteria

I have two files and would need to filter out records based on certain criteria, these column are of variable lengths, but the lengths are uniform throughout all the records of the file. I have shown a sample of three records below. Line 1-9 is the item number "0227546_1" in the case of the first... (15 Replies)
Discussion started by: MIA651
15 Replies
DateTime::Span(3pm)					User Contributed Perl Documentation				       DateTime::Span(3pm)

NAME
DateTime::Span - Datetime spans SYNOPSIS
use DateTime; use DateTime::Span; $date1 = DateTime->new( year => 2002, month => 3, day => 11 ); $date2 = DateTime->new( year => 2003, month => 4, day => 12 ); $set2 = DateTime::Span->from_datetimes( start => $date1, end => $date2 ); # set2 = 2002-03-11 until 2003-04-12 $set = $set1->union( $set2 ); # like "OR", "insert", "both" $set = $set1->complement( $set2 ); # like "delete", "remove" $set = $set1->intersection( $set2 ); # like "AND", "while" $set = $set1->complement; # like "NOT", "negate", "invert" if ( $set1->intersects( $set2 ) ) { ... # like "touches", "interferes" if ( $set1->contains( $set2 ) ) { ... # like "is-fully-inside" # data extraction $date = $set1->start; # first date of the span $date = $set1->end; # last date of the span DESCRIPTION
"DateTime::Span" is a module for handling datetime spans, otherwise known as ranges or periods ("from X to Y, inclusive of all datetimes in between"). This is different from a "DateTime::Set", which is made of individual datetime points as opposed to a range. There is also a module "DateTime::SpanSet" to handle sets of spans. METHODS
o from_datetimes Creates a new span based on a starting and ending datetime. A 'closed' span includes its end-dates: $span = DateTime::Span->from_datetimes( start => $dt1, end => $dt2 ); An 'open' span does not include its end-dates: $span = DateTime::Span->from_datetimes( after => $dt1, before => $dt2 ); A 'semi-open' span includes one of its end-dates: $span = DateTime::Span->from_datetimes( start => $dt1, before => $dt2 ); $span = DateTime::Span->from_datetimes( after => $dt1, end => $dt2 ); A span might have just a beginning date, or just an ending date. These spans end, or start, in an imaginary 'forever' date: $span = DateTime::Span->from_datetimes( start => $dt1 ); $span = DateTime::Span->from_datetimes( end => $dt2 ); $span = DateTime::Span->from_datetimes( after => $dt1 ); $span = DateTime::Span->from_datetimes( before => $dt2 ); You cannot give both a "start" and "after" argument, nor can you give both an "end" and "before" argument. Either of these conditions will cause the "from_datetimes()" method to die. To summarize, a datetime passed as either "start" or "end" is included in the span. A datetime passed as either "after" or "before" is excluded from the span. o from_datetime_and_duration Creates a new span. $span = DateTime::Span->from_datetime_and_duration( start => $dt1, duration => $dt_dur1 ); $span = DateTime::Span->from_datetime_and_duration( after => $dt1, hours => 12 ); The new "end of the set" is open by default. o clone This object method returns a replica of the given object. o set_time_zone( $tz ) This method accepts either a time zone object or a string that can be passed as the "name" parameter to "DateTime::TimeZone->new()". If the new time zone's offset is different from the old time zone, then the local time is adjusted accordingly. If the old time zone was a floating time zone, then no adjustments to the local time are made, except to account for leap seconds. If the new time zone is floating, then the UTC time is adjusted in order to leave the local time untouched. o duration The total size of the set, as a "DateTime::Duration" object, or as a scalar containing infinity. Also available as "size()". o start o end First or last dates in the span. It is possible that the return value from these methods may be a "DateTime::Infinite::Future" or a "DateTime::Infinite::Past"xs object. If the set ends "before" a date $dt, it returns $dt. Note that in this case $dt is not a set element - but it is a set boundary. o start_is_closed o end_is_closed Returns true if the first or last dates belong to the span ( begin <= x <= end ). o start_is_open o end_is_open Returns true if the first or last dates are excluded from the span ( begin < x < end ). o union o intersection o complement Set operations may be performed not only with "DateTime::Span" objects, but also with "DateTime::Set" and "DateTime::SpanSet" objects. These set operations always return a "DateTime::SpanSet" object. $set = $span->union( $set2 ); # like "OR", "insert", "both" $set = $span->complement( $set2 ); # like "delete", "remove" $set = $span->intersection( $set2 ); # like "AND", "while" $set = $span->complement; # like "NOT", "negate", "invert" o intersects o contains These set functions return a boolean value. if ( $span->intersects( $set2 ) ) { ... # like "touches", "interferes" if ( $span->contains( $dt ) ) { ... # like "is-fully-inside" These methods can accept a "DateTime", "DateTime::Set", "DateTime::Span", or "DateTime::SpanSet" object as an argument. SUPPORT
Support is offered through the "datetime@perl.org" mailing list. Please report bugs using rt.cpan.org AUTHOR
Flavio Soibelmann Glock <fglock@gmail.com> The API was developed together with Dave Rolsky and the DateTime Community. COPYRIGHT
Copyright (c) 2003-2006 Flavio Soibelmann Glock. All rights reserved. This program is free software; you can distribute it and/or modify it under the same terms as Perl itself. The full text of the license can be found in the LICENSE file included with this module. SEE ALSO
Set::Infinite For details on the Perl DateTime Suite project please see <http://datetime.perl.org>. perl v5.12.4 2011-08-22 DateTime::Span(3pm)
All times are GMT -4. The time now is 09:19 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy