This makes no sense.
No matter how debug is set, that script would dump the contents of every line read showing how many fields are present and what each field contains. The output you've shown us indicates that /data/log/test.log contains one or more blank lines, but not the data you showed us in post #19 in this thread, which was:
Please be sure that /data/log/test.log contains the text shown above, verify that the gawk invocation in modified_gawk.sh is:
and try running the script again.
If it still doesn't show lots of debugging output, show us the output from the command:
Hi sharingsunshine, Please use the 9 lines of data you showed us in your post #19 in this thread; not 53 lines from some other file! And, please put back the line of code shown in red in post #22 that you removed. Since you aren't using data that we can compare to a known input, we have no idea what is going wrong. Either the dates in the file you supplied were in June, or the calculations performed by gawk are off by a month. I'm also guessing that since the timestamps calculated by your script (before calling gawk) are showing timezone EDT, although the 5th field is currently -0400], it will be -0500] when daylight savings time is not in effect (and we will need to adjust the time calculations in gawk to account for the offset from GMT).
I am trying to compare known timestamps (in the data in the 9 lines shown in post #19 and repeated in post #22) against the calculations being performed by gawk. When you use different data, and don't show us the date and time data that is being processed, I can't determine what needs to be fixed.
Ok. So we now know that the gawk script is seeing the date and time 02/Jul/2015:01:55:57 but is generating a seconds since the Epoch value that corresponds to the date and time Tue Jun 2 01:55:57 EDT 2015.
Try changing the following line in your script:
to:
and run it again.
With any luck, this should work for you. Then you need to search your log files for a few log entries that that were created when daylight savings time was not in effect. Do you still have any log files that were created before daylight savings time went into effect this year? They should be easy to find with:
If the fgrep found any entries like that, they won't be included in the counts using your current script. If fgrep didn't find anything, you need to determine if that is because you don't have any log entries that old, or if something else is changing the date format for those entries. If any lines were found, sanitize two or three of them and add them to the file /data/log/test.log and show them to us so we can devise a time range to select one or two of them.
My guess would be that you'll need to change the line:
to one of the two following lines:
to reliably process all of your input for the US Eastern time zone, but we'll need a couple of sample lines to verify that it does works correctly for both daylight savings time and standard time.
this is what I get with the first change
I don't find anything with the fgrep but looking at the archives I don't have any files that old. Since I don't have any log files that old I put in your time zone changes to test their effects.
Here is the output I get running the fgrep commands
changing to
I get
I apologize for misleading you with the fgrep command. But, we're making great progress! If you want to look for log entries from November 2, 2014 to March 8, 2015:
should work if you have logs that cover that period. But, whether it finds anything or not, try just changing the 1st three lines of /data/log/test.log from:
to:
To shorten the debugging log, you can also delete the last three lines from that file.
And then try running the script again with either:
or:
instead of:
and using the command line:
Last edited by Don Cragun; 07-05-2015 at 01:35 AM..
Reason: Suggest shortening the sample data file to get less debugging detail.
Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp.
awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file
150318 23:19:04 logentries
150318 23:29:04 logentries
150318... (11 Replies)
Hi Friends,
I have the following logfile.
i want to make a script for calculate time by time2 - time1
1600266278|random|1|2014-09-19 02:08:56.024|2014-09-19 02:08:59.398|A|B|ROOM|Num0208559970111101788|1|dog|dos
1600266200|random|4|2014-09-19 02:08:06.572|2014-09-19... (2 Replies)
Hi Folks,
Need a clarification on files with date and time stamp.
Here is my requirement. There is a file created everyday with the following format "file.txt.YYYYMMDDHHMMSS".
Now i need to check for this file and if it is available then i need to do some task to the file.
I tried... (6 Replies)
I want to check given time stamp is between the given time stamp or not. I am using AIX.
YYYYMMDDHHMMSS
abc.csv
START TIME, END TIME
20130209018000,20130509022000
20120209018000,20130509022000
20120209018000,20130509022000
Script will check given time stamp is between above two range or... (2 Replies)
Hi,
I need help to read file in a directory on basis of time stamp.
e.g. If file access in last 2 minutes it should not be copy to remote directory.
Below is my script.
+++++++++++++++++++++++++
#!/bin/ksh
DATE=`date +"%Y-%m-%d_%H%M"`
SEPARATER=" "
exec < out_interfaces.cfg... (1 Reply)
Hi All,
PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated.
1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th
2.... (3 Replies)
Hello Friends
I am facing a weird problem :confused:, we receive thousands of files in my system on a daily basis, access time stamp on some of the files are being updated as old time stamp like 1968-01-19, Could some one help me what could be causing this? so that i can narrow down the problem... (4 Replies)
Hi,
while running the perl script i am getting this error message ,
Day '' out of range 1..31 at rsty.sh line 44
what do iam missing in the script, any suggestion
#!/usr/bin/perl
use Time::Local;
my $wday = $ARGV;
my $month = $ARGV;
# convert the month shortname into 0-11 number
if... (4 Replies)
I copied a file from one host to another using sftp. But after copying the time stamp is not updating . Even though I checked the permission, it looks good. I copied the same file to some temporary location, there it updating the time stamp. Anyone have any idea on this (6 Replies)
Hi All,
I know the timestamp of a file. Now i would like to list all the files in the with the same time stamp in the same file.
Any help would be appreciated.
Thanks.
sunny (1 Reply)