I changed the test to the same times you specified
Code:
./gawk.sh "28 May 2015" 09:33:56 09:34:21
and it is still putting this out with debug=1
Code:
1432814821 not between 1432820036 and 1432820061
1432814822 not between 1432820036 and 1432820061
1432814855 not between 1432820036 and 1432820061
1432814856 not between 1432820036 and 1432820061
1432814856 not between 1432820036 and 1432820061
1432814877 not between 1432820036 and 1432820061
1432814878 not between 1432820036 and 1432820061
1432814886 not between 1432820036 and 1432820061
1432814887 not between 1432820036 and 1432820061
1432814896 not between 1432820036 and 1432820061
1432814897 not between 1432820036 and 1432820061
1432814899 not between 1432820036 and 1432820061
1432814901 not between 1432820036 and 1432820061
1432814915 not between 1432820036 and 1432820061
1432814951 not between 1432820036 and 1432820061
1432814952 not between 1432820036 and 1432820061
1432814953 not between 1432820036 and 1432820061
1432814953 not between 1432820036 and 1432820061
1432814954 not between 1432820036 and 1432820061
1432814954 not between 1432820036 and 1432820061
1432814954 not between 1432820036 and 1432820061
1432814954 not between 1432820036 and 1432820061
I hope this is how you wanted me to test it.
I tested for gnu date and it is installed and so is Gawk.
Everything appears to be working as intended I'd suggest turning off debug and removing the extra logging code I got you to add as the actual output is probably getting lost in all the debug.
Ensure your date/time range is correct (use 24hour times or AM/PM). If still nothing try a big date/time range eg:
The big time range worked as needed. So what does that tell us?
Thanks,
It tells us that your awk script did not find any timestamps for July 2, 2015 between 1:50am and 2:30:15am in the files /data/log/access_bhp.log, /data/log/access_hpc.log, /data/log/access_tfl.log, and /data/log/access_thp.log. Can you show us any lines from any of those files with timestamps in that range that would lead you to believe that the script is working incorrectly?
I left bingbot's ip but the 1.1.1.1 represents our ip running the sitemap cron and the 2.2.2.2 is a visitor of some kind. I picked that specific time because using cloudwatch I could see my cpu was getting hammered by bots.
The awk on my system doesn't have the gawk date and time processing extensions you're using, so I can't take your code and your sample data and try it on my own.
Please save the data you showed us in post #19 in this thread in a file named /tmp/test.log and save the following slightly modified version of your script (the changes are marked in red):
Code:
#!/bin/bash
if (( $# < 3 || $# > 4 ))
then
printf "Usage: $0 from_date from_time [to_date] to_time\n" >&2
exit 2
fi
FDAY=$1
FTIME=$2
if (( $# == 3 ))
then
TDAY=$FDAY
TTIME=$3
else
TDAY=$4
TTIME=$3
fi
FROM=$(date -d "$FDAY $FTIME" +%s)
(($? != 0 )) && exit 3
TO=$(date -d "$TDAY $TTIME" +%s)
(($? != 0 )) && exit 4
if (( $# == 3 && TO < FROM ))
then
#FROM time later that TO time so add a day
(( TO+=3600*24))
fi
if (( TO < FROM ))
then
echo "$0: FROM date must be before TO date" >&2
exit 5
fi
echo "Examining from $(date -d @$FROM) ($FROM)"
echo " to $(date -d @$TO) ($TO)"
echo
FILES=/data/log/access_*.log
gawk -v F=$FROM -v T=$TO -v debug=1 '
{for(i=1;i<=NF;i++) printf "$%d=%s\n", i, $i }
FNR==1 {
for(ip in C) printf "%7d %s\n", C[ip], ip
delete C
print "Processing " FILENAME " file"
}
$5 == "-0400]" {
split($4,v,"[[/: ]")
mnum=int(index("JanFebMarAprMayJunJulAugSepOctNovDec", v[3])/3)
tm=mktime(v[4] " " mnum " " v[2] " " v[5] " " v[6] " " v[7]) + 0 if(debug) print "mtkime(" v[4] " " mnum " " v[2] " " v[5] " " v[6] " " v[7] "): " tm
if (tm >= F && tm <= T) C[$1]++
else if(debug) print tm " not between " F " and " T
}
END {for(ip in C) printf "%7d %s\n", C[ip], ip} ' /tmp/test.log
in a file named modified_gawk.sh, make it executable, and run it using the command line:
Code:
./modified_gawk.sh "02 Jul 2015" 01:55:58 01:56
and show us (in CODE tags) the output it produces.
This will allow us to see the actual seconds since the Epoch that gawk is calculating for known timestamps and hopefully will allow us to see what is going wrong without being overwhelmed with debugging data from large files containing lots of entries we don't care about.
Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp.
awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file
150318 23:19:04 logentries
150318 23:29:04 logentries
150318... (11 Replies)
Hi Friends,
I have the following logfile.
i want to make a script for calculate time by time2 - time1
1600266278|random|1|2014-09-19 02:08:56.024|2014-09-19 02:08:59.398|A|B|ROOM|Num0208559970111101788|1|dog|dos
1600266200|random|4|2014-09-19 02:08:06.572|2014-09-19... (2 Replies)
Hi Folks,
Need a clarification on files with date and time stamp.
Here is my requirement. There is a file created everyday with the following format "file.txt.YYYYMMDDHHMMSS".
Now i need to check for this file and if it is available then i need to do some task to the file.
I tried... (6 Replies)
I want to check given time stamp is between the given time stamp or not. I am using AIX.
YYYYMMDDHHMMSS
abc.csv
START TIME, END TIME
20130209018000,20130509022000
20120209018000,20130509022000
20120209018000,20130509022000
Script will check given time stamp is between above two range or... (2 Replies)
Hi,
I need help to read file in a directory on basis of time stamp.
e.g. If file access in last 2 minutes it should not be copy to remote directory.
Below is my script.
+++++++++++++++++++++++++
#!/bin/ksh
DATE=`date +"%Y-%m-%d_%H%M"`
SEPARATER=" "
exec < out_interfaces.cfg... (1 Reply)
Hi All,
PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated.
1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th
2.... (3 Replies)
Hello Friends
I am facing a weird problem :confused:, we receive thousands of files in my system on a daily basis, access time stamp on some of the files are being updated as old time stamp like 1968-01-19, Could some one help me what could be causing this? so that i can narrow down the problem... (4 Replies)
Hi,
while running the perl script i am getting this error message ,
Day '' out of range 1..31 at rsty.sh line 44
what do iam missing in the script, any suggestion
#!/usr/bin/perl
use Time::Local;
my $wday = $ARGV;
my $month = $ARGV;
# convert the month shortname into 0-11 number
if... (4 Replies)
I copied a file from one host to another using sftp. But after copying the time stamp is not updating . Even though I checked the permission, it looks good. I copied the same file to some temporary location, there it updating the time stamp. Anyone have any idea on this (6 Replies)
Hi All,
I know the timestamp of a file. Now i would like to list all the files in the with the same time stamp in the same file.
Any help would be appreciated.
Thanks.
sunny (1 Reply)
