Fetch data between two dates from a file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Fetch data between two dates from a file
# 1  
Old 08-23-2012
MySQL Fetch data between two dates from a file

Hi All,

I m new to this forum & UNix too.
currently i have a requirement which can fetch data from a logfile between two dates or timestamp.

for example:
1. data from 2012 Jun to 2012 Jul
2. data from 2012 Jun to 2012 Jul 07
3. data from 2012 Jun 16 10:20 to 2012 Jul 03 10:10

Please note that the fetched data shoul contain logs of both the dates including repeating logs.

Could you please help me solve it with awk command?

Currently i am using
Code:
awk -v s="$start" -v e="$end" '$0~s,$0~e' <logfile

Its working fine if there is only single records for the destination date.
But if we have many records like
Code:
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk
2012 Jul XXXXXXX
2012 Jul ZZZZZZZ

Then for data between 2012 Jun to 2012 Jul

It is displaying only
Code:
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk

whereas i need like:
Code:
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk
2012 Jul XXXXXXX
2012 Jul ZZZZZZZ

So please help me to get the expected output.

Thanks in advance!
KD

Last edited by Franklin52; 08-23-2012 at 09:50 AM.. Reason: Please use code tags for data and code samples
# 2  
Old 08-23-2012
If you know exact date you can use this...

Code:
$ cat file1
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk
2012 Jul XXXXXXX
2012 Jul ZZZZZZZ
$ sed -n '/2012 Jun/,/2012 Jul ZZZZZZZ/p' file1
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk
2012 Jul XXXXXXX
$ sed -n '/2012 Jun/,/2012 Jul/p'  file1
2012 Jun sdfsdjfljsdjfs
2012 Jul gjdflgldfgkdfgk

# 3  
Old 08-23-2012
@pamu
The sed code contains the same bug as the original awk code ! Both of them stop after the first occurance of the end string.

Correction. The output from the first sed is not posted correctly (there should be one more output record). If the stop string is actually the whole contents of the last record for the month, the sed works. This does not however match the specification.

I can't think of a string processing method to process alphabetic date ranges. They are going to have to be converted to a numeric form and compared as YYYYMMDDHHMMSS or perhaps seconds since the Epoch.
Do you have:
Perl ?
GNU date ?

Last edited by methyl; 08-23-2012 at 10:36 AM..
# 4  
Old 08-23-2012
Quote:
Originally Posted by methyl
@pamu
The sed code contains the same bug as the original awk code ! Both of them stop after the first occurance of the end string.
Thanks methyl,Smilie

Yes...
That's why i have given two outputs so it can be cleared both advantages and disadvantages of this..Smilie
# 5  
Old 08-23-2012
Assuming that the ending patterns are adjacent (if repeated), this should work:
Code:
awk '/2012 Jun/{p=1}
!/2012 Jul/ && prev~/2012 Jul/ && p{p=0}
{prev=$0}p' file

This User Gave Thanks to elixir_sinari For This Post:
# 6  
Old 08-23-2012
@elixir_sinari
Brilliant piece of code. It should work for a three scenarios in post #1.
# 7  
Old 08-24-2012
sorry for late thanks.

But really thanks a lot every one who suggested their ideas.
The Elixir codes really worked in all the scenarios.
once again thank YouSmilieSmilie
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Fetch data from file

Hi, I am new to scripting. I have a log file and need to fetch specific logs and copy to another file. A copy of the log is like this: =============================================================== = JOB : server123#jobs1.jobstream1 = USER : andyc = Tue 08/01/17... (3 Replies)
Discussion started by: Prngp
3 Replies

2. UNIX for Beginners Questions & Answers

How to fetch specific data from a file.?

Hi , I have a file which contains 2 days logs(here it is 24 and 25) I want to list data only for date 25 fron the file. please suggest me how should i get this. file content mentioned below 17-05-24 Name Succ Fail 00:00:29 ... (5 Replies)
Discussion started by: scriptor
5 Replies

3. Shell Programming and Scripting

Help Need to fetch the required data

Hi Guys, Am in need of your help one more time on my real data. I have a file which contains more than thousand lines of data Live data shown for 4 iterations. We have more than thousand lines of data:- -------------------------------------------------------------------------- ... (4 Replies)
Discussion started by: rocky2013
4 Replies

4. Shell Programming and Scripting

Fetch Data from File using UNIX or Perl

Hello, How All are Doing today. I have a issue, I have a file which contains the data as follow <ENVELOPE><ENVELOPE_ID>TEST</ENVELOPE_ID><ENVELOPE_EXTERNAL_ID></ENVELOPE_EXTERNAL_ID><ENVELOPE_VERSION>2</ENVELOPE_VERSION><SIResourceDefaultVersion>true</SIResourceDefaultVersion><TYPE>GS... (1 Reply)
Discussion started by: adisky123
1 Replies

5. UNIX for Dummies Questions & Answers

how to fetch data in unix

Hi All, I have a file with the below data as shown. A|2|20120430 B|EMP|NAME|DEPT C|12|SARC|01 C|23||ASDD|02 D|END OF FILE I want to fetch only the records that contains C|, what is unix command to fetch this data. Thanks (5 Replies)
Discussion started by: halpavan2
5 Replies

6. Shell Programming and Scripting

Fetch the different data by searching with a same variable from a file in AIX server

Hi, I am trying to fetch the different values in an xml file by searching with the same variable in AIX Server. <name>SharedResources/Shared/JNDI/Username</name> <value>admin</value> <name>SharedResources/Shared/JNDI/Username</name> ... (1 Reply)
Discussion started by: tejastrikez
1 Replies

7. Shell Programming and Scripting

How to fetch data between two timestamps in a file using KSH

Hi, I got a requirement to fetch data between two time stamps in a big log file and grep for a word in that particular time interval of data. Here is my log looks like: 2012/04/08-14:35:56 Abcdefg 2012/04/08-14:35:56 Hijklmnophhoishfw 2012/04/08-14:35:56... (1 Reply)
Discussion started by: siri_886
1 Replies

8. UNIX for Advanced & Expert Users

fetch dates for last 36 days in format yyyy-mm-dd

can anyone please suggest me some ideas for writing a korn shell script which will go back to 36 days from current day and print each day (incremented by 1) in the format yyyy-mm-dd until the current day. Thanks Mark (7 Replies)
Discussion started by: pavan_test
7 Replies

9. Shell Programming and Scripting

How to sca a sequential file and fetch some substring data from it

Hi, I have a task where i need to scan second column of seuential file and fetch first 3 digits of that column For e.g. FOLLOWING IS THE SAMPLE FOR MY SEQUENTIAL FILE AU_ID ACCT_NUM CRNCY_CDE THHSBC001 30045678 THB THHSBC001 10154267 THB THHSBC001 ... (2 Replies)
Discussion started by: manmeet
2 Replies

10. Shell Programming and Scripting

How to fetch data from a text file in Unix

I want to fetch passwords from a common file xxxx.txt and use it in a script. Currently the password is hardcoded so this have to be changed so that password can be fetched from text file..... Please reply asap.. Thanks (4 Replies)
Discussion started by: shikhakaul
4 Replies
Login or Register to Ask a Question