Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

retrieve lines from file which fall under the given date range

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting retrieve lines from file which fall under the given date range
# 1  
Old 07-30-2008
retrieve lines from file which fall under the given date range

Hi,

I need to retrieve the lines which fall under the given date range.
eg:In a log file,i have the lines which will have the timestamp.
the input will be some date range.eg: from date:03/Jan/2008,to date:24/Jul/2008.so now i want to retrieve the lines
which have the timestamp between these 2 given date range.

log file:
-----------
[02/Jan/2008:19:37:00-20401-59-2] Process - data
[22/Jan/2008:19:37:00-20401-59-2] Process - data
[22/Mar/2008:19:37:00-20401-63-2] Process - data
[01/Jul/2008:19:37:00-20401-63-2] Process - data
[22/Jul/2008:19:37:00-20401-63-2] Process - data
[25/Jul/2008:19:37:00-20401-63-2] Process - data

result:
Lines 2,3,4 and 5 have to be retrieved. the dates are within the given input date range.
# 2  
Old 07-30-2008
A possible solution using awk :
Code:
awk -v From="03/Jan/2008" -v To="24/Jul/2008" '
function cnvDate(date   ,d) {
   split(tolower(date), d, "/");
   return sprintf("%04.4d%02.2d%02.2d", d[3], month[d[2]], d[1]);
}
BEGIN {
   FS = "[:[]";
   month["jan"]=1 ; month["feb"]=2 ; month["mar"]=3 ; month["apr"]=4 ;
   month["may"]=5 ; month["jun"]=6 ; month["jul"]=7 ; month["aug"]=8 ;
   month["sep"]=9 ; month["oct"]=10; month["nov"]=11; month["dec"]=12;
   date_from = cnvDate(From);
   date_to   = cnvDate(To);
}
{
   date = cnvDate($2)
   if (date >= date_from && date <= date_to)
      print;
}
' inputfile

Jean-Pierre
# 3  
Old 07-30-2008
I am getting the error when i run this.
awk: syntax error near line 1
awk: bailing out near line 1

As i am new to awk,could you pls explain what exactly its doing.and i want to redirect the results to a new file.
# 4  
Old 07-30-2008
Try with nawk or gawk instead of awk.

Code:
awk -v From="03/Jan/2008" -v To="24/Jul/2008"

Defines variables From and To which contain start and end dates.

Code:
function cnvDate(date   ,d) {
   split(tolower(date), d, "/");
   return sprintf("%04.4d%02.2d%02.2d", d[3], month[d[2]], d[1]);
}

This function coverts a date from 'dd/mmm/yyyy' to 'yyyymmdd'.

Code:
BEGIN {
   FS = "[:[]";
   month["jan"]=1 ; month["feb"]=2 ; month["mar"]=3 ; month["apr"]=4 ;
   month["may"]=5 ; month["jun"]=6 ; month["jul"]=7 ; month["aug"]=8 ;
   month["sep"]=9 ; month["oct"]=10; month["nov"]=11; month["dec"]=12;
   date_from = cnvDate(From);
   date_to   = cnvDate(To);
}

Initalizations :
- Input field separator ':' or '['
- Months table used by cnvDate function
- Start and end dates format yyyymmdd

Code:
{
   date = cnvDate($2)
   if (date >= date_from && date <= date_to)
      print;
}

For each input line :
- Convert date to format yyyymmdd
- Print line if date between start and end dates


Jean-Pierre.
# 5  
Old 07-31-2008
For the same query,if the input file is like this(below),I tried getting the lines by using the field separator (FS) as blank space.
I used the code like this..
BEGIN{
FS = "[ ]";
But its not working.How I can specify that it has to take the 7th field with the delimiter single space.or is there any other way.
Input file:
-----------
2008-01-02 16:21:35,182 INFO1 loginslogging - mk99263 02/Jan/2008 16:21 2008-01-22 16:21:35,182 INFO2 loginslogging - mk99263 22/Jan/2008 16:21 2008-03-22 16:21:35,182 INFO3 loginslogging - mk99263 22/Mar/2008 16:21 2008-07-01 16:21:35,182 INFO4 loginslogging - mk99263 01/Jul/2008 16:21 2008-07-22 16:21:35,182 INFO5 loginslogging - mk99263 22/Jul/2008 16:21
2008-07-25 16:21:35,182 INFO6 loginslogging - mk99263 25/Jul/2008 16:21
# 6  
Old 07-31-2008
Field separator = space (or tab)
Date field = $7
Code:
awk -v From="03/Jan/2008" -v To="24/Jul/2008" '
function cnvDate(date   ,d) {
   split(tolower(date), d, "/");
   return sprintf("%04.4d%02.2d%02.2d", d[3], month[d[2]], d[1]);
}
BEGIN {
   month["jan"]=1 ; month["feb"]=2 ; month["mar"]=3 ; month["apr"]=4 ;
   month["may"]=5 ; month["jun"]=6 ; month["jul"]=7 ; month["aug"]=8 ;
   month["sep"]=9 ; month["oct"]=10; month["nov"]=11; month["dec"]=12;
   date_from = cnvDate(From);
   date_to   = cnvDate(To);
}
{
   date = cnvDate($7)
   if (date >= date_from && date <= date_to)
      print;
}
' inputfile

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

How to search a text in file and retrieve required lines following it with UNIX command?

I have requirement to search for a text in the file and retrieve required lines that is user defined with unix command. Eg: Find the text UNIX in the below file and need to return Test 8 & Test 9 Test 1 Test 2 Test 3 Test 4 UNIX Test 5 Test 6 Test 7 Test 8 Test 9 Result can... (8 Replies)
Discussion started by: Arunkumarsak4
8 Replies

2. Shell Programming and Scripting

awk to print out lines that do not fall between range in file

In the awk below I am trying to print out those lines in file2 that are no between $2 and $3 in file1. Both files are tab-delimeted and I think it's close but currently it is printeing out the matches. The --- are not part of the files they are just to show what lines match or fall into the range... (6 Replies)
Discussion started by: cmccabe
6 Replies

3. UNIX for Beginners Questions & Answers

How bash treats literal date value and retrieve year, month and date?

Hi, I am trying to add few (say 3 days) to sysdate using - date -d '+ 3 days' +%y%m%d and it works as expected. But how to add few (say 3 days) to a literal date value and how bash treats a literal value as a date. Can we say just like in ORACLE TO_DATE that my given literal date value... (2 Replies)
Discussion started by: pointers1234
2 Replies

4. Shell Programming and Scripting

Display lines of two date range from syslog file

Hi Guys, I want to display lines from Solaris syslog file but with 2 dates range. I have some similar solution (https://www.unix.com/shell-programming-scripting/39293-grep-log-file-between-2-dates-4.html) which works fine but as you know syslog has different date format (Jan 22) so this is not... (1 Reply)
Discussion started by: prashant2507198
1 Replies

5. Shell Programming and Scripting

Extracting specific lines of data from a file and related lines of data based on a grep value range?

Hi, I have one file, say file 1, that has data like below where 19900107 is the date, 19900107 12 144 129 0.7380047 19900108 12 168 129 0.3149017 19900109 12 192 129 3.2766666E-02 ... (3 Replies)
Discussion started by: Wynner
3 Replies

6. Shell Programming and Scripting

Retrieve lines from a file in a given date range

Hey, guys! I am trying to retrieve lines from a file in a given date range. I tried using sed -n "/${SDATE}/,/${EDATE}/p" ~/webhits/$FILE | wc -l but that doesn't work if the starting or the end date do not match exactly. If both dates match, there are no problems (for example 25 March 2008 -... (5 Replies)
Discussion started by: oopcho
5 Replies

7. Programming

How can i retrieve some specific lines from a file using C

Plz tel me how to retrieve some specific set of lines from a file and store it in a char buffer.I am seperating each record by ":" 22:abc:4 hardware:cd:xyz:2 hardware:eth:abc:6 hardware:mouse:xyz:3 hardware:ram:xyz:1 23:cde:3 hardware:cd:xyz:2 hardware:eth:abc:6 hardware:ram:xyz:1 ... (3 Replies)
Discussion started by: vigneshinbox
3 Replies

8. Shell Programming and Scripting

display the file with in the date range

Hi All, I want a shell script which can display the file with in the date range. For Example I have 15 files with the following format abc_01-01-2009.txt to abc_15-01-2009.txt. Now I want to have the files between 4th of jan to 12th files. How can I acheive this. Advance... (1 Reply)
Discussion started by: fareed_do
1 Replies

9. UNIX for Dummies Questions & Answers

get Message from file within date range

Hi All, I am a java devloper putting my hands on shell scripts. Honestly it sounds coool and interesting Presently I got stuck with the following requirement. Get the messages from file. The messages in the file are as follows: date|message1 date|message2 . . . date is of... (2 Replies)
Discussion started by: ambharish
2 Replies

10. UNIX for Dummies Questions & Answers

Need to print file names in a certain date range using ls

I need to list only file names of a specific date from the command ls -lt. :confused: (2 Replies)
Discussion started by: Shamwari
2 Replies
Login or Register to Ask a Question