Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

Need help in getting the Last 30 minutes logs from the Log File

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Need help in getting the Last 30 minutes logs from the Log File
# 1  
Old 09-02-2015
Need help in getting the Last 30 minutes logs from the Log File
I have a log file with the below contents :

log_file_updated.txt :

Code:
Jul  5 03:33:06  rsyslogd: [origin software="" swVersion="" x-pid="4567" x-info=] was 
Jul  5 03:33:09 adcsdb1 rhsmd: This system is registered.
Sep  2 02:45:48 adcsdb1 UDSAgent[44247]: 2015-07-05 04:24:48.959 INFO   Worker_Thread_4032813936 Accepted connection from host <unknown> (ipaddress 10.80.116.60:40819)
Jul  5 04:24:49 adcsdb1 multipathd: dm-12: remove map (uevent)
Sep  2 02:51:49 adcsdb1 multipathd: dm-12: devmap not registered, can't remove
Jul  5 04:24:55 adcsdb1 multipathd: mpathaq: event checker exit
Jul  5 04:24:55 adcsdb1 multipathd: uevent trigger error
Jul  5 04:24:55 adcsdb1 multipathd: dm-8: remove map (uevent)

I want only those Logs from the above file which are updated in the last 30 Minutes (which have timestamp of last 30 Minutes).

I ran the below command to find the timestamp 30 minutes before and it worked fine :

Code:
$ date --date='30 minutes ago' '+%b %_d %H:%M:%S'
Sep  2 02:29:58

I ran the below command to find the current timestamp and it worked fine :
Code:
$ date '+%b %_d %H:%M:%S'
Sep  2 03:00:04

Now to search for the logs of the last 30 minutes from the file I ran the below command and it worked fine :

Code:
awk -F - '"Sep  2 02:29:58" < $1 && $1 <="Sep  2 03:00:04"' log_file_updated.txt
Sep  2 02:45:48 adcsdb1 UDSAgent[44247]: 2015-07-05 04:24:48.959 INFO   Worker_Thread_4032813936 Accepted connection from host <unknown> (ipaddress 10.80.116.60:40819)
Sep  2 02:51:49 adcsdb1 multipathd: dm-12: devmap not registered, can't remove

However when I am trying to run the below command it is giving error :

Code:
awk -F - '"($(date --date='30 minutes ago' '+%b %_d %H:%M:%S))" < $1 && $1 <="($(date '+%b %_d %H:%M:%S'))"' log_file_updated.txt
-bash: syntax error near unexpected token `)'

Can someone Please help me.
# 2  
Old 09-02-2015
you have single quotes inside a single quoted string, you need to escape those single quotes.

the date command will not run inside quotes, read Wooledge's article on quotes
# 3  
Old 09-02-2015
In addition to a missing single quote, command substitution does not occur inside single quoted strings. To get what you seem to be trying to do, try:
Code:
awk -F - -v sd="$(date -- date='30 minutes ago' '+%b %_d %H:%M:%S')" -v ed="$(date '+%b %_d %H:%M:%S')" 'sd < $1 && $1 <= ed' log_file_updated.txt

but it seems to me that just:
Code:
awk -F - -v sd="$(date -- date='30 minutes ago' '+%b %_d %H:%M:%S')" 'sd < $1' log_file_updated.txt

would be sufficient.

Note, however, that if try this close to midnight on the 1st of a month, it may fail. For example, any date in January will sort after February 1st and any date in February will sort before March 1st.
This User Gave Thanks to Don Cragun For This Post:
rahul2662
# 4  
Old 09-02-2015
Quoting is the crucial part in parameter passing into your awk script. Try:
Code:
awk -F - -vDT="$(date --date="600 minutes ago" "+%b %_d %H:%M:%S")" ' DT < $1' log_file_updated.txt
Sep  2 02:45:48 adcsdb1 UDSAgent[44247]: 2015-07-05 04:24:48.959 INFO   Worker_Thread_4032813936 Accepted connection from host <unknown> (ipaddress 10.80.116.60:40819)
Sep  2 02:51:49 adcsdb1 multipathd: dm-12: devmap not registered, can't remove

Your second date reference yields the system time, and presuming no log entry is done for the future I neglected that test.
This User Gave Thanks to RudiC For This Post:
rahul2662
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Copy last 30 minutes' contents from a log file

Hi Guys, I am writing a bash script to capture the last 30 minutes's contents from log file to a new file. This job is a scheduled job and will run every 30 minutes. The log file is db2diag.log in DB2. I am having difficulties copying the last 30 minutes's contents. Can someone please help me.... (4 Replies)
Discussion started by: naveed
4 Replies

2. Shell Programming and Scripting

Check file creation Time minutes and if file older then 5 minutes execute some stuff

Hello all, Info: System RedHat 7.5 I need to create a script that based on the creation time, if the file is older then 5 minutes then execute some stuff, if not exit. I thought to get the creation time and minutes like this. CreationTime=$(stat -c %y /tmp/test.log | awk -F" " '{ print... (3 Replies)
Discussion started by: charli1
3 Replies

3. Shell Programming and Scripting

Grep a log file for the last 5 minutes of contents every 5 minutes

Hi all, System Ubuntu 16.04.3 LTS i have the following log INFO 2019-02-07 15:13:31,099 module.py:700] default: "POST /join/8550614e-3e94-4fa5-9ab2-135eefa69c1b HTTP/1.0" 500 2042 INFO 2019-02-07 15:13:31,569 module.py:700] default: "POST /join/6cb9c452-dcb1-45f3-bcca-e33f5d450105... (15 Replies)
Discussion started by: charli1
15 Replies

4. UNIX for Beginners Questions & Answers

Searching exception keyword in all logs in last 5 minutes

Hello Folks , I am a new bie to the world of unix , what i am planning to do is the I have the location in server to which i am access through the putty and the location is /mt/ttlog/avccomn/logs/201901/19 and at this location the files are listed as show startjmsnode1.sh_03.out... (7 Replies)
Discussion started by: punpun26262626
7 Replies

5. Shell Programming and Scripting

How to extract logs between the current time and the last 15 minutes ?

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help me?? I have a log file containing this pattern: Constructor QuartzJob ... (3 Replies)
Discussion started by: puneetkhullar
3 Replies

6. Shell Programming and Scripting

Script to search for a pattern in 30 minutes from a log file

Hello All, I have to write a script which will search for diffrent patterns like "Struck" "Out of Memory" , etc from a log file in Linux box's. Now I will be executing a cron job to find out the results by executing the script once in every 30 minutes. suppose time is 14-04-29:05:31:09 So I... (3 Replies)
Discussion started by: Shubhasis Mathr
3 Replies

7. Shell Programming and Scripting

Log File Creations for every 60 minutes

Hi All, Below script will make a copy of the existing log file with the then timestamp details. I am looking to create a copy of the existing log file for every 60 minutes and when the file limit reaches to 5, the 6th copy should overwrite the first backedup file which means all the time it... (3 Replies)
Discussion started by: Upendra Bestha
3 Replies

8. Shell Programming and Scripting

Finding errors in log file only in last 10 minutes

Hi there, I have a log file that I need to check every 10 minutes to find if a specific error exists but only in that 10 minute period. The reason is that the log is quite large, and will frequently contain these errors, so I only want alerting if it in the last 10 minutes - I don't want... (3 Replies)
Discussion started by: paul_vf
3 Replies

9. Shell Programming and Scripting

Grepping the last 30 minutes of a log file...

I need to know if anyone can assist me on how to grab the last (we'll just say "x" minutes) of a log file. How do you tell the grep command without specifying an exact window of time? (So relative instead of absolute.) Thanks, Jon (2 Replies)
Discussion started by: jtelep
2 Replies

10. Shell Programming and Scripting

get last 5 minutes' log from log file?

Hi all, I have tried to figure out a way to automatically get the last 5 minutes' log content from log file, at first, my thoughts like this, sed -n "/ $(date +\%R -d "-5 min")/,$"p syslog > newfile, but quickly I found it did not work, say I have a syslog file as following, Jul 19... (5 Replies)
Discussion started by: fedora
5 Replies
Login or Register to Ask a Question