Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Pick the last one hour lines from log matching this pattern.

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Pick the last one hour lines from log matching this pattern.
# 1  
Old 08-12-2013
Linux Pick the last one hour lines from log matching this pattern.
Hello please help me on this,

pick the last one hour lines from the log, which have the prefix time format like this.

Code:
[2/5/12 10:50:12:148 EDT] log message
[2/5/12 10:50:13:324 EDT] log message

i tried to do grep, but that failed.

my code

Code:
grep '(date +['%-m/%-d/%y %H:%M:%S:%03N %Z'])' log_file_path

This checking only the current time stamp. How to get the log for last one hour from the current time, for this time format.
Last edited by Scott; 08-12-2013 at 09:07 PM.. Reason: Please use code tags...
# 2  
Old 08-12-2013
Does your date function support the --date=STRING option?

e.g.
Code:
$ date --date="date -d "2/5/12 10:50:12.148 EDT"
Sun, Feb 05, 2012 10:50:12 AM

This User Gave Thanks to Chubler_XL For This Post:
santosh2626
# 3  
Old 08-12-2013
Quote:
Originally Posted by santosh2626
Code:
grep '(date +['%-m/%-d/%y %H:%M:%S:%03N %Z'])' log_file_path

The first problem with your code is you used single quotes: everything within single quotes is treated as is, not evaluated. This will probably work, but only find the entries with the current timestamp, not the last hour:

Code:
grep "$(date +['%-m/%-d/%y %H:%M:%S:%03N %Z'])" log_file_path

You can get a timestamp from one hour before if you manipulate the TZ variable, which holds the timezone. For instance, i am on GMT+1. To get the timestamp from one hour before I'd use:

Code:
(TZ="GMT+1" ; date)    # current time
(TZ="GMT" ; date)      # one hour ago

Lastly, grep is a poor tool for what you want to achieve, because it filters single lines. What you want to match is a range of lines and for this sed is the tool of choice. Notice how the single quoted parts are ending and restarting again:

Code:
sed  -n '/^'"$(TZ=GMT ; date +['%-m/%-d/%y %H:%M:%S:%03N %Z'])"'/,$ p' log_file_path

I hope this helps.

bakunin
This User Gave Thanks to bakunin For This Post:
santosh2626
# 4  
Old 08-12-2013
Thanks in advance.
I am using kshell. And the code i displayed above also not working.
please help me in getting the lines for the correct format. Can use grep or awk command.

---------- Post updated at 08:44 PM ---------- Previous update was at 08:36 PM ----------

i have given the logfile path /hostname/log/stdout.log at the end of sed command.
getting error cant read path: No such file or directory

---------- Post updated at 08:48 PM ---------- Previous update was at 08:44 PM ----------

Thanks backunin

That worked but displayed the complete log.
Not just the hour back lines.

Please help me.
# 5  
Old 08-12-2013
You could try this, assuming your date command supports --date :

Code:
v=$(date --date "-1 hour" +"-vY=%-y -vT=%-m -vD=%-d -vH=%-H -vM=%-M")
awk -F'[[ /:]' $v '
  $4>Y ||
  $4==Y&&$2>T ||
  $4==Y&&$2==T&&$3>D ||
  $4==Y&&$2==T&&$3==D&&$5>H ||
  $4==Y&&$2==T&&$3==D&&$5==H&&$6>M {v=1}
  v' logfile

This User Gave Thanks to Chubler_XL For This Post:
santosh2626
# 6  
Old 08-12-2013
Hello chubler.
Thanks for the post/
But getting the following syntax error.

awk: =F[[ /:]
awk:^ syntax error


And will this work for the format i displayed in the log sample?
# 7  
Old 08-12-2013
Yes, tested with the format posted, some awk versions require a space between -F and parameter try (notice additional space here):

Code:
v=$(date --date "-1 hour" +"-vY=%-y -vT=%-m -vD=%-d -vH=%-H -vM=%-M")
awk -F '[[ /:]' $v '
  $4>Y ||
  $4==Y&&$2>T ||
  $4==Y&&$2==T&&$3>D ||
  $4==Y&&$2==T&&$3==D&&$5>H ||
  $4==Y&&$2==T&&$3==D&&$5==H&&$6>M {v=1}
  v' logfile

This User Gave Thanks to Chubler_XL For This Post:
santosh2626
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Grep -v lines starting with pattern 1 and not matching pattern 2

Hi all! Thanks for taking the time to view this! I want to grep out all lines of a file that starts with pattern 1 but also does not match with the second pattern. Example: Drink a soda Eat a banana Eat multiple bananas Drink an apple juice Eat an apple Eat multiple apples I... (8 Replies)
Discussion started by: demmel
8 Replies

2. Shell Programming and Scripting

Sed: printing lines AFTER pattern matching EXCLUDING the line containing the pattern

'Hi I'm using the following code to extract the lines(and redirect them to a txt file) after the pattern match. But the output is inclusive of the line with pattern match. Which option is to be used to exclude the line containing the pattern? sed -n '/Conn.*User/,$p' > consumers.txt (11 Replies)
Discussion started by: essem
11 Replies

3. Shell Programming and Scripting

Pattern from a log within last hour

I want to extract a pattern from a log file within last hour. I am doing grep "pattern" filename | tail -1 It gives the last latest pattern but not within last hour. and if the pattern found in last hour, output success or else failure. (3 Replies)
Discussion started by: santosh2626
3 Replies

4. Shell Programming and Scripting

Pattern matching...of almost same lines

Hi all I am trying to process some data sample input is like this VARIABLE : T axis TDAY TIME : 02-FEB-2004 17:54 19755. VARIABLE : quality flag FILENAME : 1900054_prof.nc Z : 41 ... (3 Replies)
Discussion started by: Akshay Hegde
3 Replies

5. Shell Programming and Scripting

How to pick only the files which are generated in one hour?

Hello Masters, I need one help. I want to copy the files which are continuously generating on one server. But this would be on hourly basis. e.g. -rw-rw-r-- 1 akore akore 0 Feb 12 03:20 test1.log -rw-rw-r-- 1 akore akore 0 Feb 12 03:42 test2.log -rw-rw-r-- 1 akore akore 0 Feb 12 04:22... (2 Replies)
Discussion started by: akore83
2 Replies

6. Shell Programming and Scripting

Finding lines matching the Pattern and their previous lines in a file

Hi, I am trying to locate the occurences of certain pattern like 'Possible network disconnect' in a text file. I can get the actual lines matching the pttern using: grep -w 'Possible network disconnect' file_name. But I am more interested in getting the timing of these events which are... (7 Replies)
Discussion started by: sagarparadkar
7 Replies

7. Shell Programming and Scripting

pattern matching lines using the date, and then joining the lines

Hi Guys, Was trying to attempt the below using awk and sed, have no luck so far, so any help would be appreciated. Current Text File: The first line has got an "\n", and the second line has got spaces/tabs then the word and "\n" TIME SERVER/CLIENT TEXT... (6 Replies)
Discussion started by: eo29
6 Replies

8. Shell Programming and Scripting

counting the lines matching a pattern, in between two pattern, and generate a tab

Hi all, I'm looking for some help. I have a file (very long) that is organized like below: >Cluster 0 0 283nt, >01_FRYJ6ZM12HMXZS... at +/99% 1 279nt, >01_FRYJ6ZM12HN12A... at +/99% 2 281nt, >01_FRYJ6ZM12HM4TS... at +/99% 3 283nt, >01_FRYJ6ZM12HM946... at +/99% 4 279nt,... (4 Replies)
Discussion started by: d.chauliac
4 Replies

9. Shell Programming and Scripting

Pattern Matching and lines after that

I have a huge file and every paragraph has a date. But I want to retrieve the paras for the last two days only. So I can grep and findout the linenum for the first line since yesterday. Now I want to display everything after that line. And I am trying to do this inside a script so the linenum is a... (4 Replies)
Discussion started by: kaushys
4 Replies

10. Shell Programming and Scripting

pattern matching over more lines

hi, i have a text file wich contains following informations: 1 Record 90 in base GUJA_2008 (Created: 2008-01-14 19:00:38, Modified: 2008-01-15 18:54:33) 1 YADM_20080101_A91645666_A91645666 4 2008/01/15/ADM.ADM/20080101.ADM.ADM.A91645666G001.jff 1 Record 91 in base GUJA_2008 (Created:... (3 Replies)
Discussion started by: trek
3 Replies
Login or Register to Ask a Question