Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Searching the lines within a range of time period in a text file

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Searching the lines within a range of time period in a text file
# 1  
Old 05-27-2010
Question Searching the lines within a range of time period in a text file
Dear All,

Please advice me, I have a text file with one field date and time like below given. I need to find out the lines whchi content the time stamp between
Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 both can be included, using awk command which could be an interactive so that I could give the initial time and final time using the variable.


Timestamp, File Time, Channel, SID, Level, Stacked, Count, Block
2633437039, 16, 1, 9001, FD, U, 11, 1578, Wed May 26 11:03:11 2010
2633437041, 18, 1, 9001, FD, U, 2, 1764, Wed May 26 11:03:13 2010
2633437043, 20, 1, 9001, FD, S, 7, 1952, Wed May 26 11:03:15 2010
2633437045, 22, 1, 9001, FD, U, 10, 2137, Wed May 26 11:03:17 2010
2633437056, 24, 1, 2271, NT, S, 5, 2323, Wed May 26 11:03:28 2010
2633437047, 24, 1, 9001, FD, U, 18, 2324, Wed May 26 11:03:19 2010
2633437049, 26, 1, 9001, FD, U, 20, 2511, Wed May 26 11:03:21 2010
2633437051, 28, 1, 9001, FD, U, 19, 2698, Wed May 26 11:03:23 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
2633437059, 36, 1, 9001, FD, U, 18, 3446, Wed May 26 11:03:31 2010
2633437060, 38, 1, 9001, FD, U, 13, 3633, Wed May 26 11:03:32 2010
2633437062, 40, 1, 9001, FD, U, 19, 3820, Wed May 26 11:03:34 2010
2633437065, 42, 1, 9001, FD, U, 13, 4007, Wed May 26 11:03:37 2010
2633437067, 44, 1, 9001, FD, U, 6, 4194, Wed May 26 11:03:39 2010
2633437068, 46, 1, 9001, FD, U, 17, 4381, Wed May 26 11:03:40 2010
2633437070, 48, 1, 9001, FD, U, 15, 4568, Wed May 26 11:03:42 2010
2633437072, 50, 1, 9001, FD, U, 15, 4755, Wed May 26 11:03:44 2010
2633437074, 52, 1, 9001, FD, U, 10, 4942, Wed May 26 11:03:46 2010
2633437076, 54, 1, 9001, FD, U, 15, 5128, Wed May 26 11:03:48 2010
2633437076, 54, 1, 9001, FD, S, 24, 5129, Wed May 26 11:03:48 2010
2633437078, 56, 1, 9001, FD, U, 16, 5315, Wed May 26 11:03:50 2010
2633437080, 58, 1, 9001, FD, U, 18, 5502, Wed May 26 11:03:52 2010
# 2  
Old 05-27-2010
Hi,

a possible solution will be the next:
create a new file where the first field is a numeric value formed from any date:
For example:
Wed May 26 11:03:11 2010
tranforms in:
2010-05-26-11-03-11 or simply: 20100526110311

Then, when all lines are transformed, execute a sort and with a script search comparing the numbers.

For me is tedious to write now, but the idea will be a solution.
# 3  
Old 05-27-2010
You can do something like:
Code:
awk 'BEGIN {
  printf "Enter the starttime <hh:mm:ss> :"
  getline start < "-"
  printf "Enter the stoptime <hh:mm:ss> :"
  getline stop < "-"
  gsub(":","",start)
  gsub(":","",stop)
}
{d=$12;gsub(":","",d)}
d >= start && d <= stop
' file

# 4  
Old 05-27-2010
Question #Searching the lines within a range of time period in a text file
Pls advice, if I need to enter the starttime and stoptime as Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 what should I follow.

Quote:
Originally Posted by Franklin52
You can do something like:
Code:
awk 'BEGIN {
  printf "Enter the starttime <hh:mm:ss> :"
  getline start < "-"
  printf "Enter the stoptime <hh:mm:ss> :"
  getline stop < "-"
  gsub(":","",start)
  gsub(":","",stop)
}
{d=$12;gsub(":","",d)}
d >= start && d <= stop
' file

# 5  
Old 05-27-2010
Quote:
Originally Posted by chinmayadalai
...
Please advice me, I have a text file with one field date and time like below given. I need to find out the lines whchi content the time stamp between
Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 both can be included, using awk command which could be an interactive so that I could give the initial time and final time using the variable.
...
No idea about awk, but here's one way to solve it with Perl:

Code:
$
$
$ # show the content of the file to be inspected
$ cat f0
Timestamp, File Time, Channel, SID, Level, Stacked, Count, Block
2633437039, 16, 1, 9001, FD, U, 11, 1578, Wed May 26 11:03:11 2010
2633437041, 18, 1, 9001, FD, U,  2, 1764, Wed May 26 11:03:13 2010
2633437043, 20, 1, 9001, FD, S,  7, 1952, Wed May 26 11:03:15 2010
2633437045, 22, 1, 9001, FD, U, 10, 2137, Wed May 26 11:03:17 2010
2633437056, 24, 1, 2271, NT, S,  5, 2323, Wed May 26 11:03:28 2010
2633437047, 24, 1, 9001, FD, U, 18, 2324, Wed May 26 11:03:19 2010
2633437049, 26, 1, 9001, FD, U, 20, 2511, Wed May 26 11:03:21 2010
2633437051, 28, 1, 9001, FD, U, 19, 2698, Wed May 26 11:03:23 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
2633437059, 36, 1, 9001, FD, U, 18, 3446, Wed May 26 11:03:31 2010
2633437060, 38, 1, 9001, FD, U, 13, 3633, Wed May 26 11:03:32 2010
2633437062, 40, 1, 9001, FD, U, 19, 3820, Wed May 26 11:03:34 2010
2633437065, 42, 1, 9001, FD, U, 13, 4007, Wed May 26 11:03:37 2010
2633437067, 44, 1, 9001, FD, U,  6, 4194, Wed May 26 11:03:39 2010
2633437068, 46, 1, 9001, FD, U, 17, 4381, Wed May 26 11:03:40 2010
2633437070, 48, 1, 9001, FD, U, 15, 4568, Wed May 26 11:03:42 2010
2633437072, 50, 1, 9001, FD, U, 15, 4755, Wed May 26 11:03:44 2010
2633437074, 52, 1, 9001, FD, U, 10, 4942, Wed May 26 11:03:46 2010
2633437076, 54, 1, 9001, FD, U, 15, 5128, Wed May 26 11:03:48 2010
2633437076, 54, 1, 9001, FD, S, 24, 5129, Wed May 26 11:03:48 2010
2633437078, 56, 1, 9001, FD, U, 16, 5315, Wed May 26 11:03:50 2010
2633437080, 58, 1, 9001, FD, U, 18, 5502, Wed May 26 11:03:52 2010
$
$ # show the content of the Perl program that processes file "f0"
$
$ cat -n f0.pl
     1  #perl -w
     2  use Date::Calc qw(Parse_Date Date_to_Time);
     3  $file = "f0";
     4
     5  # Process START_DATE that has format "Day Mon DD HH24:MI:SS YYYY"
     6  $start = $ARGV[0];
     7  if ($start =~ m/(.*) ([\d:]+) (.*)/) {
     8    @start_dt = Parse_Date("$1 $3");
     9    @start_ts = split(/:/,$2);
    10  }
    11  $stime = Date_to_Time(@start_dt, @start_ts);
    12
    13  # Process END_DATE that has format "Day Mon DD HH24:MI:SS YYYY"
    14  $end = $ARGV[1];
    15  if ($end =~ m/(.*) ([\d:]+) (.*)/) {
    16    @end_dt = Parse_Date("$1 $3");
    17    @end_ts = split(/:/,$2);
    18  }
    19  $etime = Date_to_Time(@end_dt, @end_ts);
    20
    21  # process the file
    22  open (IN, $file) or die "Can't open $file: $!";
    23  while (<IN>) {
    24    if ($. > 1) {
    25      chomp;
    26      if (/^.*, (.*) ([\d:]+) (.*)$/) {
    27        @file_dt = Parse_Date("$1 $3");
    28        @file_ts = split(/:/,$2);
    29        $ftime = Date_to_Time(@file_dt, @file_ts);
    30        print $_,"\n" if ($stime <= $ftime and $ftime <= $etime);
    31      }
    32    }
    33  }
    34  close (IN) or die "Can't open $file: $!";
$
$
$ # run the Perl program, passing <START_DATE> and <END_DATE> as parameters
$ perl f0.pl "Wed May 26 11:03:11 2010" "Wed May 26 11:03:52 2010"
2633437039, 16, 1, 9001, FD, U, 11, 1578, Wed May 26 11:03:11 2010
2633437041, 18, 1, 9001, FD, U,  2, 1764, Wed May 26 11:03:13 2010
2633437043, 20, 1, 9001, FD, S,  7, 1952, Wed May 26 11:03:15 2010
2633437045, 22, 1, 9001, FD, U, 10, 2137, Wed May 26 11:03:17 2010
2633437056, 24, 1, 2271, NT, S,  5, 2323, Wed May 26 11:03:28 2010
2633437047, 24, 1, 9001, FD, U, 18, 2324, Wed May 26 11:03:19 2010
2633437049, 26, 1, 9001, FD, U, 20, 2511, Wed May 26 11:03:21 2010
2633437051, 28, 1, 9001, FD, U, 19, 2698, Wed May 26 11:03:23 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
2633437059, 36, 1, 9001, FD, U, 18, 3446, Wed May 26 11:03:31 2010
2633437060, 38, 1, 9001, FD, U, 13, 3633, Wed May 26 11:03:32 2010
2633437062, 40, 1, 9001, FD, U, 19, 3820, Wed May 26 11:03:34 2010
2633437065, 42, 1, 9001, FD, U, 13, 4007, Wed May 26 11:03:37 2010
2633437067, 44, 1, 9001, FD, U,  6, 4194, Wed May 26 11:03:39 2010
2633437068, 46, 1, 9001, FD, U, 17, 4381, Wed May 26 11:03:40 2010
2633437070, 48, 1, 9001, FD, U, 15, 4568, Wed May 26 11:03:42 2010
2633437072, 50, 1, 9001, FD, U, 15, 4755, Wed May 26 11:03:44 2010
2633437074, 52, 1, 9001, FD, U, 10, 4942, Wed May 26 11:03:46 2010
2633437076, 54, 1, 9001, FD, U, 15, 5128, Wed May 26 11:03:48 2010
2633437076, 54, 1, 9001, FD, S, 24, 5129, Wed May 26 11:03:48 2010
2633437078, 56, 1, 9001, FD, U, 16, 5315, Wed May 26 11:03:50 2010
2633437080, 58, 1, 9001, FD, U, 18, 5502, Wed May 26 11:03:52 2010
$
$ # once again, with different parameter values
$ perl f0.pl "Wed May 26 11:03:11 2010" "Wed May 26 11:03:20 2010"
2633437039, 16, 1, 9001, FD, U, 11, 1578, Wed May 26 11:03:11 2010
2633437041, 18, 1, 9001, FD, U,  2, 1764, Wed May 26 11:03:13 2010
2633437043, 20, 1, 9001, FD, S,  7, 1952, Wed May 26 11:03:15 2010
2633437045, 22, 1, 9001, FD, U, 10, 2137, Wed May 26 11:03:17 2010
2633437047, 24, 1, 9001, FD, U, 18, 2324, Wed May 26 11:03:19 2010
$
$ # and once more
$ perl f0.pl "Wed May 26 11:03:25 2010" "Wed May 26 11:03:45 2010"
2633437056, 24, 1, 2271, NT, S,  5, 2323, Wed May 26 11:03:28 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
2633437059, 36, 1, 9001, FD, U, 18, 3446, Wed May 26 11:03:31 2010
2633437060, 38, 1, 9001, FD, U, 13, 3633, Wed May 26 11:03:32 2010
2633437062, 40, 1, 9001, FD, U, 19, 3820, Wed May 26 11:03:34 2010
2633437065, 42, 1, 9001, FD, U, 13, 4007, Wed May 26 11:03:37 2010
2633437067, 44, 1, 9001, FD, U,  6, 4194, Wed May 26 11:03:39 2010
2633437068, 46, 1, 9001, FD, U, 17, 4381, Wed May 26 11:03:40 2010
2633437070, 48, 1, 9001, FD, U, 15, 4568, Wed May 26 11:03:42 2010
2633437072, 50, 1, 9001, FD, U, 15, 4755, Wed May 26 11:03:44 2010
$
$

HTH,
tyler_durden
Last edited by durden_tyler; 05-27-2010 at 03:32 PM..
# 6  
Old 05-27-2010
Quote:
Originally Posted by chinmayadalai
Pls advice, if I need to enter the starttime and stoptime as Wed May 26 11:03:11 2010 and Wed May 26 11:03:52 2010 what should I follow.
Code:
awk '/Wed May 26 11:03:11 2010/,/Wed May 26 11:03:48 2010/' file

Adjust time stamp as required.
# 7  
Old 05-27-2010
Using danmero's idea -

Code:
$
$
$ cat f0
Timestamp, File Time, Channel, SID, Level, Stacked, Count, Block
2633437039, 16, 1, 9001, FD, U, 11, 1578, Wed May 26 11:03:11 2010
2633437041, 18, 1, 9001, FD, U,  2, 1764, Wed May 26 11:03:13 2010
2633437043, 20, 1, 9001, FD, S,  7, 1952, Wed May 26 11:03:15 2010
2633437045, 22, 1, 9001, FD, U, 10, 2137, Wed May 26 11:03:17 2010
2633437056, 24, 1, 2271, NT, S,  5, 2323, Wed May 26 11:03:28 2010
2633437047, 24, 1, 9001, FD, U, 18, 2324, Wed May 26 11:03:19 2010
2633437049, 26, 1, 9001, FD, U, 20, 2511, Wed May 26 11:03:21 2010
2633437051, 28, 1, 9001, FD, U, 19, 2698, Wed May 26 11:03:23 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
2633437059, 36, 1, 9001, FD, U, 18, 3446, Wed May 26 11:03:31 2010
2633437060, 38, 1, 9001, FD, U, 13, 3633, Wed May 26 11:03:32 2010
2633437062, 40, 1, 9001, FD, U, 19, 3820, Wed May 26 11:03:34 2010
2633437065, 42, 1, 9001, FD, U, 13, 4007, Wed May 26 11:03:37 2010
2633437067, 44, 1, 9001, FD, U,  6, 4194, Wed May 26 11:03:39 2010
2633437068, 46, 1, 9001, FD, U, 17, 4381, Wed May 26 11:03:40 2010
2633437070, 48, 1, 9001, FD, U, 15, 4568, Wed May 26 11:03:42 2010
2633437072, 50, 1, 9001, FD, U, 15, 4755, Wed May 26 11:03:44 2010
2633437074, 52, 1, 9001, FD, U, 10, 4942, Wed May 26 11:03:46 2010
2633437076, 54, 1, 9001, FD, U, 15, 5128, Wed May 26 11:03:48 2010
2633437076, 54, 1, 9001, FD, S, 24, 5129, Wed May 26 11:03:48 2010
2633437078, 56, 1, 9001, FD, U, 16, 5315, Wed May 26 11:03:50 2010
2633437080, 58, 1, 9001, FD, U, 18, 5502, Wed May 26 11:03:52 2010
$
$ awk -F", " -v FROM="Wed May 26 11:03:20 2010" -v TO="Wed May 26 11:03:30 2010" 'NR>1 && $9 > FROM && $9 < TO' f0
2633437056, 24, 1, 2271, NT, S,  5, 2323, Wed May 26 11:03:28 2010
2633437049, 26, 1, 9001, FD, U, 20, 2511, Wed May 26 11:03:21 2010
2633437051, 28, 1, 9001, FD, U, 19, 2698, Wed May 26 11:03:23 2010
2633437053, 30, 1, 9001, FD, U, 18, 2885, Wed May 26 11:03:25 2010
2633437055, 32, 1, 9001, FD, U, 21, 3072, Wed May 26 11:03:27 2010
2633437057, 34, 1, 9001, FD, U, 17, 3259, Wed May 26 11:03:29 2010
$
$

I guess danmero's script tries to perform a literal match of date+timestamps supplied (not sure about this though).

Code:
$
$ awk '/Wed May 26 11:03:20 2010/,/Wed May 26 11:03:30 2010/' f0
$

tyler_durden
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Match text to lines in a file, iterate backwards until text or text substring matches, print to file

hi all, trying this using shell/bash with sed/awk/grep I have two files, one containing one column, the other containing multiple columns (comma delimited). file1.txt abc12345 def12345 ghi54321 ... file2.txt abc1,text1,texta abc,text2,textb def123,text3,textc gh,text4,textd... (6 Replies)
Discussion started by: shogun1970
6 Replies

2. Red Hat

CPU Usage statistics Dump in a text file over a period of time

I am facing issue related to performance of one customized application running on RHEL 5.9. The application stalls for some unknown reason that I need to track. For that I require some tool or shell scripts that can monitor the CPU usage statistics (what we get in TOP or in more detail by other... (6 Replies)
Discussion started by: Anjan Ganguly
6 Replies

3. Shell Programming and Scripting

Bash script - printing range of lines from text file

I'm working on a new exercise that calls for a script that will take in two arguments on the command line (representing the range of line numbers) and will subsequently print those lines from a a specified file. Command line would look like this: ./lines_script.bash 5 15 <file.txt. The script would... (8 Replies)
Discussion started by: ksmarine1980
8 Replies

4. Shell Programming and Scripting

Extract data from log file in specified range of time

I was searching for parsing a log file and found what I need in this link http://stackoverflow.com/questions/7575267/extract-data-from-log-file-in-specified-range-of-time But the most useful answer (posted by @Kent): # this variable you could customize, important is convert to seconds. # e.g... (2 Replies)
Discussion started by: kingk110
2 Replies

5. Shell Programming and Scripting

Grep in a log file within a time range (hour)

Hi, im trying to write a grep script that returns me the last inputs added in the last hour in the log file. Literally i have nothing yet but: grep 'Line im looking for' LOGFILE.log | tail -1 this only gives me the last input, but no necessarily from the last hour. Help Please. (4 Replies)
Discussion started by: blacksteel1988
4 Replies

6. Shell Programming and Scripting

To get the Files between Time Period

All, How to get the list of files through a unix command which exists / created / updated between 8 PM to 11:59 PM from a particular location. Regards Oracle User (3 Replies)
Discussion started by: Oracle_User
3 Replies

7. Shell Programming and Scripting

searching a file with a specified text without using conventional file searching commands

without using conventional file searching commands like find etc, is it possible to locate a file if i just know that the file that i'm searching for contains a particular text like "Hello world" or something? (5 Replies)
Discussion started by: arindamlive
5 Replies

8. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

9. UNIX for Dummies Questions & Answers

Need help in searching a file by range

hi, i need to search a file by range, the file (f3.txt) contains: x1=0123318739 x2=0120123456 x3=0120453576 x4=0110445654 x5=0120432343 x6=0129423 x7=0104323433 x8=01232132134 x9=0122344242 x10=012006196 x11=012016546 x12=012091235 x13=0121064598 x14=012194562 x15=0122028556... (3 Replies)
Discussion started by: takyeldin
3 Replies

10. UNIX for Dummies Questions & Answers

remove lines in text starting with . (period)

how can i remove lines from a text file starting with . (a period) (11 Replies)
Discussion started by: Movomito
11 Replies
Login or Register to Ask a Question