Bash Script Closest Timestamp (LOG FILE)


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Bash Script Closest Timestamp (LOG FILE)
# 1  
Old 09-13-2015
Bash Script Closest Timestamp (LOG FILE)

Hi All,
I need something where I could take the date from FILE1 and in this case the date is Thu Sep 10 13:48:42 EDT 2015 and I need it to match the closest PREVIOUS date in the log file FILE2 this means in this specific case it would be Thu Sep 10 2015 13:35:28.

I only need the closest date and nothing else as I am grepping and sedding with the value gotten, btw the date can be days before in some rare cases.


FILE1
Code:
77777;HP-BLADE;4;661;0;1;Thu Sep 10 2015 13:48:42;0;0;Matthew;send email to Carlos;




FILE2
Code:
Thu Sep 10 2015 03:01:18;.1.3.6.1.4.1.x.x.3.34;Critical;Status Events;87.200.x.x;77777;Automation Device;ERROR DETECTED;HP-BLADE
Thu Sep 10 2015 10:29:48;.1.3.6.1.4.1.x.x.3.34;Critical;Status Events;87.200.x.x;77777;Automation Device;ERROR DETECTED;HP-BLADE
Thu Sep 10 2015 13:35:28;.1.3.6.1.4.1.x.x.3.34;Critical;Status Events;87.200.x.x;77777;Automation Device;ERROR DETECTED;HP-BLADE
Thu Sep 10 2015 16:34:49;.1.3.6.1.4.1.x.x.3.34;Critical;Status Events;87.200.x.x;77777;Automation Device;ERROR DETECTED;HP-BLADE


So the preferred output I am looking for is only the date example:
Thu Sep 10 2015 13:35:28



Thanks again all for all your help, this has been driving me nuts!! and I'm sure there is a simple solution. I would prefer a 1 liner and if possible in awk and sed if possible if not I don't mind calling a third party script.

Last edited by Don Cragun; 09-13-2015 at 05:07 AM.. Reason: Add CODE and ICODE tags.
# 2  
Old 09-13-2015
A little more information would help:
  1. What operating system are you using?
  2. What shell are you using?
  3. For single digit day of month values, is there a leading space or a leading zero? (I.e., does the code that writes the date stamp use %d or %e to format the day of month?)
  4. Is the date & time in the first file always in the 7th semicolon delimited field?
  5. Is the date & time in the second file always in the 1st semicolon delimited field?
  6. Are the date & time values in the second file always in increasing time order (as in your sample FILE2 contents)?
  7. And, just to be clear, if you find the same date & time in both files, the matching entry in the 2nd file would NOT be selected; is that correct? I.e., you really mean the date & time in the 2nd file must be earlier, not earlier or the same time, than the date & time in the 1st file.
# 3  
Old 09-13-2015
Hi Don,
Thank you for the fast reply below are the answers to your questions.


What operating system are you using?
Answer: Debian

What shell are you using?
Answer: Bash

For single digit day of month values, is there a leading space or a leading zero? (I.e., does the code that writes the date stamp use %d or %e to format the day of month?)
Answer: Correct it is a 0 so for example the 9th would be 09.


Is the date & time in the first file always in the 7th semicolon delimited field?
Answer: Correct it is always on the 7th column / semicolon

Is the date & time in the second file always in the 1st semicolon delimited field?
Answer: Correct it is always on the 1st column / semicolon

Are the date & time values in the second file always in increasing time order (as in your sample FILE2 contents)?
Answer: Correct

And, just to be clear, if you find the same date & time in both files, the matching entry in the 2nd file would NOT be selected; is that correct? I.e., you really mean the date & time in the 2nd file must be earlier, not earlier or the same time, than the date & time in the 1st file.

Answer: It would never be the same time on the first file and the time would always be more than the second file as these are comments added after the value in FILE2 are added. Which means if both times are matching FILE1 would take the previous date/time and not the equal one which means it would always be "earlier" pick.

Thanks again Don for all your help I greatly appreciate all your help!

Sincerely,
Roberto
# 4  
Old 09-13-2015
Would this help (assuming Debian's date is GNU date):
Code:
IFS=";" read _ _ _ _ _ _ DT _ < file1
DTS=$(date -d"$DT" +%s)
tac file2 |
while IFS=";" read LG _
  do [ $(date -d"$LG" +%s) -lt $DTS ] \
     && { echo $LG; break; }
  done
Thu Sep 10 2015 13:35:28

Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Bash script for looking in log file

Hello, I'm a beginner in shell scripting. I would really appreciate some help from the forum. I want to write a small script that will look in apache error log. If it finds the appropriate word. It would execute some commands. In my case the apache error log is situated in:... (2 Replies)
Discussion started by: ajaysingh99
2 Replies

2. Shell Programming and Scripting

How can i make my cron/script to generate a log filename with timestamp in it ?

Hello Friends, I would like my script to display date timestamps in the file name for every script execution. Below is the scenario: (just for testing purpose) I scheduled a cron job, lets say it runs every 5 min and record/logs output in to a log file. 0,5,10,15,20,25,30,35,40,45,50,55 *... (5 Replies)
Discussion started by: System Admin 77
5 Replies

3. Shell Programming and Scripting

Find the closest value in another csv file preceding it and following it?

Hi, Is this possible? I want to take a csv file and find the closest value in another csv file preceding it and following it. For ex. In this csv file, I'll take the first line: 1309341156.800000000 1309341156.802500000 1309341156.805000000 1309341156.807500000 and find the closest... (2 Replies)
Discussion started by: superbbrr
2 Replies

4. Shell Programming and Scripting

Increment and find closest value until the end of the file is reached.

Hi all, I have a file which looks like: 0 1.1985506 1 1.2237930 2 1.2159038 3 1.2668828 4 1.2650216 5 1.2474344 6 1.2817688 7 1.2721698 8 1.2665005 9 1.2826315 10 1.2797879 11 1.3201736 12 1.3116595 13 1.3361583 14 1.3309238 (2 Replies)
Discussion started by: ezitoc
2 Replies

5. Shell Programming and Scripting

bash script to create txt files based on timestamp

Hi , please guide me for a bash script that will create a txt files and the name of the txt files will be as of timestamp so that each file name will be different from other and these files will be get created say after every 10 minutes in a folder(/home/p2000/sxs137), please guide me how would... (1 Reply)
Discussion started by: nks342
1 Replies

6. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

7. Shell Programming and Scripting

Logging ALL standard out of a bash script to a log file, but still show on screen

Is it possible to store all standard-out of a bash script and the binaries it calls in a log file AND still display the stdout on screen? I know this is possible to store ALL stdout/stderr of a script to a single log file like: exec 1>&${logFile} exec 2>&1 But running a script with the... (3 Replies)
Discussion started by: ckmehta
3 Replies

8. Shell Programming and Scripting

concatenate log file lines up to timestamp

Hi, Using sed awk or perl I am trying to do something similar to https://www.unix.com/shell-programming-scripting/105887-sed-awk-concatenate-lines-until-blank-line-2.html but my requirement is slightly different. What I am trying to accomplish is to reformat a logfile such that all lines... (4 Replies)
Discussion started by: AlanC
4 Replies

9. Linux

SED/AWK Script to clear log file using timestamp?

I have a log file on our system which fills up with lines that have been timestamped, as follows.... 03/03/2008 10:56:06:815] (ERROR) balance: continuing session to genapp02 : 18500 03/03/2008 10:56:06:820] (ERROR) balance: continuing session to genapp02 : 18500 03/03/2008 10:56:07:003]... (2 Replies)
Discussion started by: davesimm
2 Replies

10. UNIX for Dummies Questions & Answers

Spooling a log file with timestamp

Hi From shell script i am invoking sqlplus to connect to oracle database and then i spool a csv file as with output. What i want to do is to change the file name with timestamp on it so after spooling finish shell script change file name with time stamp. can someone help me to do that . Thanks... (2 Replies)
Discussion started by: ukadmin
2 Replies
Login or Register to Ask a Question