grep based on date


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting grep based on date
# 1  
Old 01-23-2009
grep based on date

Hello! I have a circular log file which contains data that looks like this:
01/23/09 08:24:19:04 treadle_data = L3^M
01/23/09 08:24:19:09 STRIP 3 LOW
01/23/09 08:24:19:09 treadle_data = L4^M
01/23/09 08:24:19:09 STRIP 4 LOW
01/23/09 08:24:19:09 treadle_data = FF^M
01/23/09 08:24:19:09 FORWARD
01/23/09 08:24:21:20 treadle_data = H1^M
01/23/09 08:24:21:20 STRIP 1 HIGH
01/23/09 08:24:21:20 treadle_data = H2^M
01/23/09 08:24:21:20 STRIP 2 HIGH
01/23/09 08:24:21:20 treadle_data = H3^M
01/23/09 08:24:21:20 STRIP 3 HIGH
01/23/09 08:24:21:25 treadle_data = H4^M
01/23/09 08:24:21:25 STRIP 4 HIGH
01/23/09 08:24:21:25 treadle_data = L1^M
01/23/09 08:24:21:26 STRIP 1 LOW
01/23/09 08:24:21:26 treadle_data = L2^M
01/23/09 08:24:21:26 STRIP 2 LOW
01/23/09 08:24:21:31 treadle_data = L3^M
01/23/09 08:24:21:31 STRIP 3 LOW
01/23/09 08:24:21:31 treadle_data = L4^M
01/23/09 08:24:21:31 STRIP 4 LOW
01/23/09 08:24:21:31 treadle_data = FF^M
01/23/09 08:24:21:31 FORWARD
I want to be able to run `grep -c FORWARD` only on lines whose date is greater than a date I specify. So supposed I created a function called "cntFW". If I typed "cntFW 01/23/09 08:24:21:00", then the function would only return 1 based on the data given above. Hope I am clear. Can this be done? Heeeelp.
# 2  
Old 01-23-2009
You will need some kind of filter - perl or C or python, probably. Assuming you do not have GNU date--
Here is a C example:
Code:
/* time filter: tfilter.c usage:  "date time" must be quoted
       tfilter "date time" [filename]
       source command | tfilter "date time"
*/
#include <stdlib.h>
#include <stdio.h>
#include <time.h>
#include <string.h>

double to_msec(const char *src)
{
    struct tm tm={0,0,0,0,0,0,0,0};
    time_t retval=0;
    char ms[4]={0x0};
    char dest[32]={0x0};
    
    ms[0]='.';
    memcpy(&ms[1], &src[18], 2);
    memcpy(dest, src, 20);
	if(strptime(dest, "%D %H:%M:%S", &tm)==NULL)
		return retval;
	retval=mktime(&tm);
	sprintf(dest, "%u", retval);
	strcat(dest, ms);
	return atof(dest);
}

int main(int argc, char **argv)
{
	char tmp[2048]={0x0};
	FILE *in=(argc>2) ? fopen(argv[2], "r") : stdin;
	char working[32]={0x0};
	const double compar=to_msec(argv[1]);

	if(in==NULL) { perror("input file error"); exit(1);}
	while (fgets(tmp, sizeof(tmp), in)!=NULL)
	{
		if(to_msec(tmp) > compar)
			printf("%s", tmp);
	}
    return 0;
}

Code:
tfilter "01/23/09 08:24:21:20" filename

output:
Code:
01/23/09 08:24:21:25 treadle_data = H4^M
01/23/09 08:24:21:25 STRIP 4 HIGH
01/23/09 08:24:21:25 treadle_data = L1^M
01/23/09 08:24:21:26 STRIP 1 LOW
01/23/09 08:24:21:26 treadle_data = L2^M
01/23/09 08:24:21:26 STRIP 2 LOW
01/23/09 08:24:21:31 treadle_data = L3^M
01/23/09 08:24:21:31 STRIP 3 LOW
01/23/09 08:24:21:31 treadle_data = L4^M
01/23/09 08:24:21:31 STRIP 4 LOW
01/23/09 08:24:21:31 treadle_data = FF^M
01/23/09 08:24:21:31 FORWARD

Code:
tfilter "01/23/09 08:24:21:20" filename | grep 'FORWARD'

output:
Code:
01/23/09 08:24:21:31 FORWARD


Last edited by vgersh99; 01-23-2009 at 05:25 PM.. Reason: fixed code tag
# 3  
Old 01-26-2009
is there any way to do this with only shell scripting?
# 4  
Old 01-26-2009
What OS and platform do you have?
# 5  
Old 01-26-2009
sco unix
# 6  
Old 01-26-2009
i have a feeling that I can use the AWK for this, but I am not familiar enough.
# 7  
Old 01-26-2009
One fairly possible way to do this in shell.
-- download the GNU date utility which can turn a date & time into epoch seconds
After you install it read the man page and see how to get GNU date to input a date & time and output epoch seconds.

Some shells have extended date support like zsh - you can check into that as well.

The basic idea is:
turn the search parameter into epoch seconds, append the millisecond value
loop thru the file, turn the leading sdat/time into epoch seconds + ms.
Compare the epoch seconds value to the search parameter epoch seconds value. If it is greater ( or equal to or greater than) the search epoch seconds print it - pipe it to grep 'FORWARD'

Another approach is to use Perderabo's datecalc routine in our FAQ - the thread is 'Date Arithmetic' The script does use perl.
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Splitting week start date and end date based on custom period start dates

Below are my custom period start and end dates based on a calender, these dates are placed in a file, for each period i need to split into three weeks for each period row, example is given below. Could you please help out to achieve solution through shell script.. File content: ... (2 Replies)
Discussion started by: nani2019
2 Replies

2. HP-UX

HP/UX command to pull file name/date based on date

HI, Can anyone tell me how to pull the date and file name separated by a space using the find command or any other command. I want to look through several directories and based on a date timeframe (find -mtime -7), output the file name (without the path) and the date(in format mmddyyyy) to a... (2 Replies)
Discussion started by: lnemitz
2 Replies

3. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

4. Shell Programming and Scripting

Script to determine Date,TotalFile,total size of file based on date

I have file listed like below -rw-r--r--+ 1 test test 17M Nov 26 14:43 test1.gz -rw-r--r--+ 1 test test 0 Nov 26 14:44 test2.gz -rw-r--r--+ 1 test test 0 Nov 27 10:41 test3.gz -rw-r--r--+ 1 test test 244K Nov 27 10:41 test4.gz -rw-r--r--+ 1 test test 17M Nov 27 10:41 test5.gz I... (5 Replies)
Discussion started by: krish2014
5 Replies

5. UNIX for Dummies Questions & Answers

Copy log based on from-date and to-date

Hi all, i go a customer support requirement where i need to scan several files based on from/to date like 1-oct to 2-oct please help... (3 Replies)
Discussion started by: AbhiJ
3 Replies

6. UNIX for Dummies Questions & Answers

Condition based on Timestamp (Date/Time based) from logfile (Epoch seconds)

Below is the sample logfile: Userids Date Time acb Checkout time: 2013-11-20 17:00 axy Checkout time: 2013-11-22 12:00 der Checkout time: 2013-11-17 17:00 xyz Checkout time: 2013-11-19 16:00 ddd Checkout time: 2013-11-21 16:00 aaa Checkout... (9 Replies)
Discussion started by: asjaiswal
9 Replies

7. Shell Programming and Scripting

Calculating expiry date using date,sed,grep

Hi, I would greatly appreciate it if someone can help me with my problem. I have a crawler which collects spam URLs everyday & this data needs to be published in a blacklist. Here's the catch: The "Time To Live" (TTL) for each URL is 3 months (or whatever for that matter). If i see the... (5 Replies)
Discussion started by: r4v3n
5 Replies

8. Shell Programming and Scripting

Get the oldest date based on date in the filename

I am using ksh93 on Solaris. Ok, this may seem like a simple request at first. I have a directory that contains sets of files with a YYYYMMDD component to the name, along with other files of different filespecs. something like this: 20110501_1.dat 20110501_2.dat 20110501_3.dat... (2 Replies)
Discussion started by: gary_w
2 Replies

9. Shell Programming and Scripting

grep and cat based on specific mod date

I'm trying to figure out how to open and copy all contents of files last modded on aug 14 to one single text file. Also, I'm trying to do this in one command string. I have ls -l -R | grep "Aug 1" but all this does is print the -l info with Aug 1 in it. how can I modify this so that ls... (3 Replies)
Discussion started by: mmixology
3 Replies

10. Shell Programming and Scripting

grep latest file based on date.

hi all, not sure if this has been posted b4 but i try to search but not valid. this is my question: when i do a ls -ltr there will be a list generated as follows: -rw-r--r-- 1 root sys 923260 Jan 10 04:38 FilePolling.41025.083TL021.xml -rw-r--r-- 1 root sys 1761337 Jan 10 04:40... (12 Replies)
Discussion started by: lweegp
12 Replies
Login or Register to Ask a Question