Selecting A Section of A File Based On the Time Within It


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Selecting A Section of A File Based On the Time Within It
# 1  
Old 01-25-2010
Selecting A Section of A File Based On the Time Within It

Hi,

I have a file formated like this:

Code:
John      7.22    2010-01-25_17:01:36
George   8.22    2010-01-25_17:02:36
Bob        9.62    2010-01-25_17:04:36
Jane       10.11  2010-01-25_17:05:36
Emma      4.52   2010-01-25_17:01:36

What I want to do is cut out only the entries that have been made in the last fifteen minutes. I had this:

Code:
###Finding Current Time
time_now=$(perl -e 'use POSIX qw(strftime);
use Time::Local;
$back_time = (time() - (1*0));
$title_date = strftime("%Y-%m-%d_%H:%M", localtime($back_time));
print $title_date;')

###Finding Time 15 Mins Ago
time_minus_fifteen=$(perl -e 'use POSIX qw(strftime);
use Time::Local;
$back_time = (time() - (1*900));
$title_date = strftime("%Y-%m-%d_%H:%M", localtime($back_time));
print $title_date;')

####Selecting 15 min section of file
awk "/$time_minus_fifteen/,/$time_now/" my_file > fifteen_min_file

This worked nicely until there were no transactions in a particular minute the awk bit was looking for e.g. if it was

Code:
awk "/2010-01-25_16:51/,/2010-01-25_17:07/" my_file > fifteen_min_file

and there were no entries made with the timestamp 16:51 then the awk part of the script fails.


Can anybody think of a better way to do this? Is there any function that allows you to query a timestamp. For example in SQL you could do something like time between 16:50:00 and 17:05:00. Is there something like this in awk, ksh or perl?

Any help appreciated!

Cheers
# 2  
Old 01-28-2010
Code:
use POSIX qw/strftime/;
use Date::Calc qw(Delta_YMDHMS);

my $curr_time = strftime('%Y-%m-%d_%H:%M:%S',localtime);

while(<>){
        chomp;
        my ($name,$val,$date) = split("\t");
        print "$name\t$val\t$date\n" if (get_diff_in_dates($date,$curr_time));
}


sub get_diff_in_dates {
        my ($date1,$date2) = @_;
        my ($ymd1,$hms1) = split("_",$date1);
        my ($ymd2,$hms2) = split("_",$date2);
        my ($diff_year,$diff_mon,$diff_day,$diff_hr,$diff_min,$diff_sec) = Delta_YMDHMS(split("-",$ymd1),split(":",$hms1),split("-",$ymd2),split(":",$hms2));
        ($diff_min <= 15) ? return 1 : return 0;
}


HTH,
PL
# 3  
Old 01-28-2010
if you have control of the log file
i would include the unix epoch time as a field, then it would be trivial.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Selecting section and removing match

I have a file with contents as shown in file.texi Would like to keep only the sections that have inlineifset till the empty line is reached. Finally replace the following string with a space @inlineifset{mrg, @opar{@bullet{} I had written the following command but it messed my file ... (6 Replies)
Discussion started by: Danette
6 Replies

2. UNIX for Beginners Questions & Answers

Selecting lines based on the value in the 3rd column.

Hello, I have a sample data like this: A1 B1 100.00 B1 A1 100.00 A2 B2 90.80 B2 A2 90.80 A3 B3 99.07 B3 A3 99.07 A4 B4 99.00 B4 A4 99.00 A5 B5 97.13 B5 A5 99.53 . . Ax By i By Ax j each two lines are same comparison with opposite order. What I expected is... (3 Replies)
Discussion started by: nengcheng
3 Replies

3. Shell Programming and Scripting

Copying section of file based on search criteria

Hi Guru's, I am new to unix scripting. I have a huge file with user details in it(file2) and I have another file with a list of users(file1). Script has to search a user from file1 and get all the associated lines from file2. Example: fiel1: cn=abc cn=DEF cn=xyx File 2: dn:... (10 Replies)
Discussion started by: Samingla
10 Replies

4. Shell Programming and Scripting

Selecting sequences based on scores

I have two files with thousands of sequences of different lengths. infile1 contains the actual sequences and infile2 the scores for each A, T, G and C in infile1. Something like this: infile1: >HZVJKYI01ECH5R TTGATGTGCCAGCTGCCGTTGGTGTGCCAA >HZVJKYI01AQWJ8 GGATATGATGATGAACTGGTTTGGCACACC... (4 Replies)
Discussion started by: Xterra
4 Replies

5. Shell Programming and Scripting

Extract record from file based on section.

input file output file (1 Reply)
Discussion started by: lathigara
1 Replies

6. Shell Programming and Scripting

Selecting rows from a pipe delimited file based on condition

HI all, I have a simple challenge for you.. I have the following pipe delimited file 2345|98|1809||x|969|0 2345|98|0809||y|0|537 2345|97|9809||x|544|0 2345|97|0909||y|0|651 9685|98|7809||x|321|0 9685|98|7909||y|0|357 9685|98|7809||x|687|0 9685|98|0809||y|0|234 2315|98|0809||x|564|0 ... (2 Replies)
Discussion started by: nithins007
2 Replies

7. Shell Programming and Scripting

Deleting a section based on search from other file

Hi Everyone, I need some help to accomplish the below. help is highly appriciated. I have a 45 mb file with ldap entries. Each user entry is separated by a string # entry-id: 1 and so on. Some of the entries has a string xyz: true. I want to delete the section if the user section has xyz: true... (6 Replies)
Discussion started by: Samingla
6 Replies

8. Shell Programming and Scripting

Extract section of file based on word in section

I have a list of Servers in no particular order as follows: virtualMachines="IIBSBS IIBVICDMS01 IIBVICMA01"And I am generating some output from a pre-existing script that gives me the following (this is a sample output selection). 9/17/2010 8:00:05 PM: Normal backup using VDRBACKUPS... (2 Replies)
Discussion started by: jelloir
2 Replies

9. Shell Programming and Scripting

selecting only few lines from many based on a common pattern

Hi, I have a file having some thousand records with the following sort of lines: Error: Failed to get order data Order: PO-BBBTGZE Error: No CLI Error: Failed to get order data Order: PO-SBDJUZA Order: PO-XBBIDEN Error: No CLI Error: Failed to get order data Order: PO-BBDJUTQ Order:... (2 Replies)
Discussion started by: damansingh
2 Replies

10. Shell Programming and Scripting

sed & awk--get section of file based 2 params

I need to get a section of a file based on 2 params. I want the part of the file between param 1 & 2. I have tried a bunch of ways and just can't seem to get it right. Can someone please help me out.....its much appreciated. Here is what I have found that looks like what I want....but doesn't... (12 Replies)
Discussion started by: Andy Cook
12 Replies
Login or Register to Ask a Question