This makes no sense.
No matter how debug is set, that script would dump the contents of every line read showing how many fields are present and what each field contains. The output you've shown us indicates that /data/log/test.log contains one or more blank lines, but not the data you showed us in post #19 in this thread, which was:
./modified_gawk.sh "02 Jul 2015" 01:55:57 01:55:58
Examining from Thu Jul 2 01:55:57 EDT 2015 (1435816557)
to Thu Jul 2 01:55:58 EDT 2015 (1435816558)
Processing /data/log/test.log file
mtkime(2015 6 02 01 55 00): 1433224500
1433224500 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 00): 1433224500
1433224500 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 03): 1433224503
1433224503 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 03): 1433224503
1433224503 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 04): 1433224504
1433224504 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 06): 1433224506
1433224506 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 07): 1433224507
1433224507 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 09): 1433224509
1433224509 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 09): 1433224509
1433224509 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 12): 1433224512
1433224512 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 13): 1433224513
1433224513 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 15): 1433224515
1433224515 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 16): 1433224516
1433224516 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 17): 1433224517
1433224517 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 17): 1433224517
1433224517 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 18): 1433224518
1433224518 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 19): 1433224519
1433224519 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 21): 1433224521
1433224521 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 22): 1433224522
1433224522 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 24): 1433224524
1433224524 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 25): 1433224525
1433224525 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 27): 1433224527
1433224527 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 28): 1433224528
1433224528 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 30): 1433224530
1433224530 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 31): 1433224531
1433224531 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 33): 1433224533
1433224533 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 33): 1433224533
1433224533 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 36): 1433224536
1433224536 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 37): 1433224537
1433224537 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 39): 1433224539
1433224539 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 40): 1433224540
1433224540 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 42): 1433224542
1433224542 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 43): 1433224543
1433224543 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 45): 1433224545
1433224545 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 46): 1433224546
1433224546 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 48): 1433224548
1433224548 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 49): 1433224549
1433224549 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 51): 1433224551
1433224551 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 52): 1433224552
1433224552 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 54): 1433224554
1433224554 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 55): 1433224555
1433224555 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 57): 1433224557
1433224557 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 57): 1433224557
1433224557 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 58): 1433224558
1433224558 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 57): 1433224557
1433224557 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 59): 1433224559
1433224559 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 59): 1433224559
1433224559 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 59): 1433224559
1433224559 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 59): 1433224559
1433224559 not between 1435816557 and 1435816558
mtkime(2015 6 02 01 55 59): 1433224559
1433224559 not between 1435816557 and 1435816558
mtkime(2015 6 02 02 01 55): 1433224915
1433224915 not between 1435816557 and 1435816558
Hi sharingsunshine, Please use the 9 lines of data you showed us in your post #19 in this thread; not 53 lines from some other file! And, please put back the line of code shown in red in post #22 that you removed. Since you aren't using data that we can compare to a known input, we have no idea what is going wrong. Either the dates in the file you supplied were in June, or the calculations performed by gawk are off by a month. I'm also guessing that since the timestamps calculated by your script (before calling gawk) are showing timezone EDT, although the 5th field is currently -0400], it will be -0500] when daylight savings time is not in effect (and we will need to adjust the time calculations in gawk to account for the offset from GMT).
I am trying to compare known timestamps (in the data in the 9 lines shown in post #19 and repeated in post #22) against the calculations being performed by gawk. When you use different data, and don't show us the date and time data that is being processed, I can't determine what needs to be fixed.
Ok. So we now know that the gawk script is seeing the date and time 02/Jul/2015:01:55:57 but is generating a seconds since the Epoch value that corresponds to the date and time Tue Jun 2 01:55:57 EDT 2015.
With any luck, this should work for you. Then you need to search your log files for a few log entries that that were created when daylight savings time was not in effect. Do you still have any log files that were created before daylight savings time went into effect this year? They should be easy to find with:
Code:
fgrep '-0500]' /data/log/*.log
If the fgrep found any entries like that, they won't be included in the counts using your current script. If fgrep didn't find anything, you need to determine if that is because you don't have any log entries that old, or if something else is changing the date format for those entries. If any lines were found, sanitize two or three of them and add them to the file /data/log/test.log and show them to us so we can devise a time range to select one or two of them.
My guess would be that you'll need to change the line:
to reliably process all of your input for the US Eastern time zone, but we'll need a couple of sample lines to verify that it does works correctly for both daylight savings time and standard time.
I don't find anything with the fgrep but looking at the archives I don't have any files that old. Since I don't have any log files that old I put in your time zone changes to test their effects.
Here is the output I get running the fgrep commands
I apologize for misleading you with the fgrep command. But, we're making great progress! If you want to look for log entries from November 2, 2014 to March 8, 2015:
Code:
fgrep ' -0500]' /data/log/*.log
should work if you have logs that cover that period. But, whether it finds anything or not, try just changing the 1st three lines of /data/log/test.log from:
Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp.
awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file
150318 23:19:04 logentries
150318 23:29:04 logentries
150318... (11 Replies)
Hi Friends,
I have the following logfile.
i want to make a script for calculate time by time2 - time1
1600266278|random|1|2014-09-19 02:08:56.024|2014-09-19 02:08:59.398|A|B|ROOM|Num0208559970111101788|1|dog|dos
1600266200|random|4|2014-09-19 02:08:06.572|2014-09-19... (2 Replies)
Hi Folks,
Need a clarification on files with date and time stamp.
Here is my requirement. There is a file created everyday with the following format "file.txt.YYYYMMDDHHMMSS".
Now i need to check for this file and if it is available then i need to do some task to the file.
I tried... (6 Replies)
I want to check given time stamp is between the given time stamp or not. I am using AIX.
YYYYMMDDHHMMSS
abc.csv
START TIME, END TIME
20130209018000,20130509022000
20120209018000,20130509022000
20120209018000,20130509022000
Script will check given time stamp is between above two range or... (2 Replies)
Hi,
I need help to read file in a directory on basis of time stamp.
e.g. If file access in last 2 minutes it should not be copy to remote directory.
Below is my script.
+++++++++++++++++++++++++
#!/bin/ksh
DATE=`date +"%Y-%m-%d_%H%M"`
SEPARATER=" "
exec < out_interfaces.cfg... (1 Reply)
Hi All,
PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated.
1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th
2.... (3 Replies)
Hello Friends
I am facing a weird problem :confused:, we receive thousands of files in my system on a daily basis, access time stamp on some of the files are being updated as old time stamp like 1968-01-19, Could some one help me what could be causing this? so that i can narrow down the problem... (4 Replies)
Hi,
while running the perl script i am getting this error message ,
Day '' out of range 1..31 at rsty.sh line 44
what do iam missing in the script, any suggestion
#!/usr/bin/perl
use Time::Local;
my $wday = $ARGV;
my $month = $ARGV;
# convert the month shortname into 0-11 number
if... (4 Replies)
I copied a file from one host to another using sftp. But after copying the time stamp is not updating . Even though I checked the permission, it looks good. I copied the same file to some temporary location, there it updating the time stamp. Anyone have any idea on this (6 Replies)
Hi All,
I know the timestamp of a file. Now i would like to list all the files in the with the same time stamp in the same file.
Any help would be appreciated.
Thanks.
sunny (1 Reply)
