Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

To print lines between 2 timestamps using awk|sed and regex

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting To print lines between 2 timestamps using awk|sed and regex
# 1  
Old 04-07-2013
Hammer & Screwdriver To print lines between 2 timestamps using awk|sed and regex
Hi,

I am using the following code to fetch lines that are generated in last 1 hr . Hence, I am using date function to calculate -last 1 hr & the current hr and then somehow use awk (or sed-if someone could guide me better)
with some regex pattern.
Code:
dt_1=`date +%h" "%d", "%Y\ %l -d  "1 hour ago"`  #Apr 05, 2013 1   --- if ran at 02 AM
dt_2=`date +%p -d  "1 hour ago"`  # AM|PM
echo $dt_1
echo $dt_2
dt_3=`date +%h" "%d", "%Y\ %l`
dt_4=`date +%p`
echo $dt_3
echo $dt_4
awk  -v a="{$dt_1}" -v b="{$dt_2}" -v c="{$dt_3}" -v d="{$dt_4}" '$0~/a:[0-9][0-9]:[0-9][0-9] b/{flag=1;next} $0~/c:[0-9][0-9]:[0-9][0-9] d/{flag=0} flag' /some/logfile >> /some/dump_logs.txt

The above code doesnot give any errors and I have also tried without $0~

Let's say I want all the lines starting from
Apr 07, 2013 1:mm:ss AM
to
Apr 07, 2013 2:mm:ss AM

if script gets executed b/w 2:00-2:59 hrs

OR
lines between
Apr 07, 2013 12:mm:ss PM
to
Apr 07, 2013 1:mm:ss PM

if script gets executed b/w 13:00-13:59 hrs

I have been searching for similar posts reported by various users, but I can only see variables being used in awk, without any regex pattern.

Could anyone please guide me here ?
# 2  
Old 04-07-2013
change
Code:
$0~/a:[0-9][0-9]:[0-9][0-9] b/

to
Code:
$0~ (a ":[0-9][0-9]:[0-9][0-9] "b)

(and the same of the others)
# 3  
Old 04-07-2013
Code:
awk  -v a="{$dt_1}" -v b="{$dt_2}" -v c="{$dt_3}" -v d="{$dt_4}" '$0~ (a ":[0-9][0-9]:[0-9][0-9] "b){flag=1;next} $0~ (c ":[0-9][0-9]:[0-9][0-9] "d){flag=0} flag' /some/logfile >> /some/dump_logs.txt

I used the above snippet, as suggested...but it still does not work...

Am I missing something ..some space ??
# 4  
Old 04-07-2013
In what way does it not work? What does it do?
# 5  
Old 04-07-2013
The entire script given in my first post, when I execute it...(with this updated awk statement) , simply echoes the date patterns.

'dump_logs.txt' file is of 0 bytes, after executing the script .
# 6  
Old 04-07-2013
Here is what I tried and it worked:
Code:
awk -v DT1="${dt_1}:[0-9][0-9]:[0-9][0-9] ${dt_2}" -v DT2="${dt_3}:[0-9][0-9]:[0-9][0-9] ${dt_4}" ' {
        n = match ( $0, DT1 )
        if ( n )
                flag = 1
        n = match ( $0, DT2 )
        if ( n )
                flag = 0
} flag ' logfile

Input File:
Code:
$ cat logfile
line1
Apr 07, 2013  5:00:00 PM
line3
line4
Apr 07, 2013  6:00:00 PM
line6
line7

Output:
Code:
$ ./sarah
Apr 07, 2013  5:00:00 PM
line3
line4

# 7  
Old 04-07-2013
I think the problem lies with the calculation of $dt_1 and $dt_3
Code:
dt_1=`date +%h" "%d", "%Y\ %l -d  "1 hour ago"`
 
dt_3=`date +%h" "%d", "%Y\ %l`

The space between Y and l, is the culprit Smilie
I had given this space to accomodate 2 digit hour. For 1 digit hr, if there is no space between Y and l, the script works fine.
Otherwise, it does nothing..(as happening with me till now)

Your script worked because there are 2 spaces between the year and the hr in the log file that you have shown:
2013 5
and so it could match exactly.
Smilie
If you could suggest something here....that would be much appreciated.
How should I modify my date format calculation to match in exact date in log file - bearing in mind 1 digit and 2 digit hr
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sed/awk to delete a regex between range of lines

Hi Guys I am looking for a solution to one problem to remove parentheses in a range of lines. Input file module bist_logic_inst(a, ab , dhd, dhdh , djdj, hdh, djjd, jdj, dhd, dhp, dk ); input a; input ab; input dhd; input djdj; input dhd; output hdh; output djjd; output jdj;... (5 Replies)
Discussion started by: kshitij
5 Replies

2. Shell Programming and Scripting

(n)awk: print regex search output lines in one line

Hello. I have been looking high and low for the solution for this. I seems there should be a simple answer, but alas. I have a big xml file, and I need to extract certain information from specific items. The information I need can be found between a specific set of tags. let's call them... (2 Replies)
Discussion started by: Tobias-Reiper
2 Replies

3. UNIX for Advanced & Expert Users

sed REGEX to print multiple occurrences of a pattern from a line

I have a line that I need to parse through and extract a pattern that occurs multiple times in it. Example line: getInfoCall: info received please proceed, getInfoCall: info received please proceed, getInfoCall: info received please proceed, getInfoCall: info received please proceed,... (4 Replies)
Discussion started by: Vidhyaprakash
4 Replies

4. UNIX for Advanced & Expert Users

awk print all fields except matching regex

grep -v will exclude matching lines, but I want something that will print all lines but exclude a matching field. The pattern that I want excluded is '/mnt/svn' If there is a better solution than awk I am happy to hear about it, but I would like to see this done in awk as well. I know I can... (11 Replies)
Discussion started by: glev2005
11 Replies

5. Shell Programming and Scripting

Print lines that match regex on xth string

Hello, I need an awk command to print only the lines that match regex on xth field from file. For example if I use this command awk -F"|" ' $22 == "20130117090000.*" 'It wont work, I think, because single quotes wont allow the usage of the metacharacter star * . On the other hand I dont know... (2 Replies)
Discussion started by: black_fender
2 Replies

6. Shell Programming and Scripting

Print lines between two strings multiple occurencies (with sed, awk, or grep)

Hello, I can extract lines in a file, between two strings but only one time. If there are multiple occurencies, my command show only one block. Example, monfichier.txt contains : debut_sect texte L1 texte L2 texte L3 texte L4 fin_sect donnees inutiles 1 donnees inutiles 2 ... (8 Replies)
Discussion started by: theclem35
8 Replies

7. Shell Programming and Scripting

How to print the lines between the pattern using awk/grep/sed?

Hi, I need a help to search a pattern and print the multiple lines between them. Input file: Tue May 29 12:30:33 EDT 2012:threadWebContainer : 357:com.travimp.hotelierlinks.abba.service.RequestHandler.requestService(String, ITICSDataSet): hotelCancelReservation request: ... (4 Replies)
Discussion started by: aroragaurav.84
4 Replies

8. Shell Programming and Scripting

Print lines after regex

Hello, I need to print four lines inmediatly after the regexp, but not the line containing the regexp. The print should show the four lines together in one. Thanks! (13 Replies)
Discussion started by: auratus42
13 Replies

9. Shell Programming and Scripting

print first few lines, then apply regex on a specific column to print results.

abc.dat tty cpu tin tout us sy wt id 0 0 7 3 19 71 extended device statistics r/s w/s kr/s kw/s wait actv wsvc_t asvc_t %w %b device 0.0 133.2 0.0 682.9 0.0 1.0 0.0 7.2 0 79 c1t0d0 0.2 180.4 0.1 5471.2 3.0 2.8 16.4 15.6 15 52 aaaaaa1-xx I want to skip first 5 line... (4 Replies)
Discussion started by: kchinnam
4 Replies

10. Shell Programming and Scripting

sed - print only matching regex

Hi folks, Lets say I have the following text file: name, lastname, 1234, name.lastname@test.com name1, lastname1, name2.lastname2@test.com, 2345 name, 3456, lastname, name3.lastname3@test.com 4567, name, lastname, name4.lastname4@test.com I now need the following output: 1234... (5 Replies)
Discussion started by: domi55
5 Replies
Login or Register to Ask a Question