Processing a log file based on date/time input and the date/time on the log file


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Processing a log file based on date/time input and the date/time on the log file
# 1  
Old 03-15-2008
Processing a log file based on date/time input and the date/time on the log file

Hi,

I'm trying to accomplish the following and would like some suggestions or possible bash script examples that may work

I have a directory that has a list of log files that's periodically dumped from a script that is crontab that are rotated 4 generations. There will be a time stamp that is associated when the file is created.

application.log.1
application.log.2
application.log.3
application.log.4
application.log.5

I have another script that will take an input of date/time, I'm not sure which is the easiest format to compare, from the short look around it sounds like epoch is the easiest/fastest.

I would like to take that format whichever it is, and basically subtract 5minutes which is a poll period thats been defined and then use that date/time to compare with the logs that are rotated and basically process the log that closely matches the "currentInputDateTime" where it can not be larger than "currentInputDateTime" but less than/equal to the current time, I assume this might be a "ls -tr | awk '{print $6 $7}' to get the date/time and somehow convert those to epoch.

I was wondering if this was something that has been encountered before so the wheel does not have to be re-invented.

Thanks in advance
# 2  
Old 03-15-2008
Here you can probably find something useful:

https://www.unix.com/answers-frequent...rithmetic.html

Regards
# 3  
Old 03-16-2008
I was taking a look at those links and I saw that you could pass in a parameter such as the following:

date -d "2008-03-15 11:40" --date="5 minutes ago" +%F==%I:%M

Though the output was:

2008-03-15==09:30

I was expecting to get back 11:35 for the time, is there something I'm missing to take a date format and subtract 5minutes from that time and convert it to epoch time to compare with other epoch values?
# 4  
Old 03-16-2008
It should be something like:

Code:
date --date "2008-03-15 11:40 5 min ago" +%F==%I:%M

But what is the reason for using this format "+%F==%I:%M"?
You can get the epoch time of 5 minutes ago with:

Code:
date --date "2008-03-15 11:40 5 min ago" +%s

Regards
# 5  
Old 03-16-2008
Thank you, I was going down that route of using the "5 minutes ago" but didn't realize it was support to be part of the input string, that was where I got confused on and I did not couldn't find an example online or was not reading the man pages correctly. I'll try what you've provided with my script to see if I can get this working
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Shell Script | Parse log file after a given date and time stamp

I am developing one script which will take log file name, output file name, date, hour and minute as an argument and based on these inputs, the script will scan and capture all the error(s) that have been triggered from a given time. Example: script should capture all the error after 13:50 on Jan... (2 Replies)
Discussion started by: ROMA3
2 Replies

2. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

3. Shell Programming and Scripting

Extracting log files based on date and time.

Hi All, i have some log files generated in a folder daily with the format abc.def.20130306.100001 ghi.jkl.20130306.100203 abc.def.20130305.100001 ghi.jkl.20130305.100203 the format is the date followed by time . all i want is to get the files that are generated for todays... (3 Replies)
Discussion started by: mahesh300182
3 Replies

4. Shell Programming and Scripting

Delete log files content older than 30 days and append the lastest date log file date

To delete log files content older than 30 days and append the lastest date log file date in the respective logs I want to write a shell script that deletes all log files content older than 30 days and append the lastest log file date in the respective logs This is my script cd... (2 Replies)
Discussion started by: sreekumarhari
2 Replies

5. Shell Programming and Scripting

Grep the Content of a LOG File which has latest Date and Time

Hi All, Need a small help. I have a log file which keeps updating for every Minute with multiple number of lines. I just want to grep few properties which has latest Date and Time to it. How do i do it? I wanted to grep a property by name "Reloading cache with a maximum of" from the... (4 Replies)
Discussion started by: nvindraneel
4 Replies

6. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

7. Shell Programming and Scripting

Extract info from log file and compute using time date stamp

Looking for a shell script or a simple perl script . I am new to scripting and not very good at it . I have 2 directories . One of them holds a text file with list of files in it and the second one is a daily log which shows the file completion time. I need to co-relate both and make a report. ... (0 Replies)
Discussion started by: breez_drew
0 Replies

8. UNIX for Dummies Questions & Answers

Adding date and time to a log file

Morning all Im hoping you can help me. We have a nice new oracle server :( and are needing to move some files around for EDI and BACS. The server runs windows but has an app called MKS toolkit installed which give unix commands. (Needed for the oracle stuff) I have had a go using dos commands... (2 Replies)
Discussion started by: ltodd2
2 Replies

9. Shell Programming and Scripting

Date and time log file

Hi, I wrote a small perl script in unix that searches in a file and saves some information in a separate file. Since this is a log file, I would like to have the date added to file name. I have no idea where to start. output: log_010907.txt thanks ken (8 Replies)
Discussion started by: captoro
8 Replies

10. UNIX for Dummies Questions & Answers

Inserting Date&Time Stamp In Existing Log File

I am trying to insert a line with a date stamp in a file that is used to monitor activity in one of our directories. By doing this, I want to grep that file each day and go to the last entry for each time a error occurred and pull all errors generated if any exist. If error exists I want that error... (3 Replies)
Discussion started by: shephardfamily
3 Replies
Login or Register to Ask a Question