Login or Register to Ask a Question and Join Our Community


Shell Programming and Scripting

sed/awk date range?

Login to Discuss or Reply to this Discussion in Our Community

 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting sed/awk date range?
# 1  
Old 10-17-2011
sed/awk date range?
Hi,

I am trying to grep out a date range in an access log file. I defined the date like so;

Code:
DATE1=$(date --date '1 hour ago' '+%m/%d/%y:%H:%M:%S')
DATE2=$(date '+%m/%d/%y:%H:%M:%S')

Then I just used cat to get the hits to the url into a results.txt;

Code:
touch /tmp/results.txt
cat /var/log/httpd/access_log | grep index.php >> /tmp/results.txt

How would I use sed/awk to get the exact entries for the date ranges that were defined?

Thanks for any help.

Cheers!
# 2  
Old 10-17-2011
see thread :

https://www.unix.com/shell-programmin...#post302565304
# 3  
Old 10-17-2011
I saw that post as well, but when I try what is suggested, I just get an empty tmp.log, there should be at least a few lines.

Here is the script I wrote;

Code:
date1=$(date --date '1 hour ago' '+%m/%d/%y:%H:%M:%S')
date2=$(date '+%m/%d/%y:%H:%M:%S')

cat /var/log/httpd/access_log | grep index.php >> results.txt

awk -v d1="${date1}" -v d2="${date2}" '$0~d1{p=1} $0~d2{p=0} p' results.txt >> tmp.log

# 4  
Old 10-17-2011
How does your results.txt look like ?

Give us a cat ... Smilie
# 5  
Old 10-17-2011
results.txt is just the grep'd access_log for apache on my proof of concept VM;

Code:
127.0.0.1 - - [17/Oct/2011:12:06:15 -0700] "GET /cacti/include/main.css HTTP/1.1" 304 - "http://localhost/cacti/index.php" "Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.9.168 Version/11.51"
127.0.0.1 - - [17/Oct/2011:12:06:15 -0700] "GET /cacti/images/favicon.ico HTTP/1.1" 304 - "http://localhost/cacti/index.php" "Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.9.168 Version/11.51"
127.0.0.1 - - [17/Oct/2011:12:06:15 -0700] "GET /cacti/include/layout.js HTTP/1.1" 304 - "http://localhost/cacti/index.php" "Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.9.168 Version/11.51"
127.0.0.1 - - [17/Oct/2011:12:06:15 -0700] "GET /cacti/images/shadow_gray.gif HTTP/1.1" 304 - "http://localhost/cacti/index.php" "Opera/9.80 (X11; Linux x86_64; U; en) Presto/2.9.168 Version/11.51"

From there I need to just get the entries over the last hour. So, do I have to use awk and filter everything but the numbers out, then use egrep to get the correct range and get the line count from that?

---------- Post updated at 02:52 PM ---------- Previous update was at 01:08 PM ----------

From what I have been reading, I would have to convert the date to be fully numeric, then sed would work nicely to get a range. Not sure how I can covert the log file, adjusting the httpd.conf logging format isn't an option.

Suggestions?
# 6  
Old 10-17-2011
something to start with working on your 'grep-ed' file sample:
nawk -f epx.awk myGreppedLogFile
epx.awk:
Code:
BEGIN {
 FS="[[ ]"
 mon="JAN|FEB|MAR|APR|MAY|JUN|JUL|AUG|SEP|OCT|NOV|DEC"
   monN=split(mon, monA, "|");
   for(i=1; i<=monN; i++) {
     monA[monA[i]]=i;
     delete monA[i];
   }
}
{
    n=split($5,a, "[/:]")
    printf("%s ->[%s%02d%02d%s%s%s]\n", $5, a[3], monA[toupper(a[2])], a[1], a[4], a[5], a[6])
}

You don't need sed/grep - do it all natively in awk.
# 7  
Old 10-17-2011
Than you very much. That converted the dates nicely. What do you suggest for getting the entries of the last hour? Current time, going back 60 minutes. I used date to mimic the format and going back 1 hour. I tried using sed but it returns 0.

Code:
sed -n '/$DATE1/,/$DATE2/p' output.log | wc -l

That look right?
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. UNIX for Beginners Questions & Answers

Sed/awk to delete a regex between range of lines

Hi Guys I am looking for a solution to one problem to remove parentheses in a range of lines. Input file module bist_logic_inst(a, ab , dhd, dhdh , djdj, hdh, djjd, jdj, dhd, dhp, dk ); input a; input ab; input dhd; input djdj; input dhd; output hdh; output djjd; output jdj;... (5 Replies)
Discussion started by: kshitij
5 Replies

2. Shell Programming and Scripting

sed or awk to remove specific column to one range

I need to remove specific column to one range source file 3 1 000123456 2 2 000123569 3 3 000123564 12 000123156 15 000125648 128 000125648 Output required 3 000123456 2 000123569 3 000123564 12 000123156 15 000125648 128 000125648 (6 Replies)
Discussion started by: ranjancom2000
6 Replies

3. Shell Programming and Scripting

Get range out using sed or awk, only if given pattern match

Input: START OS:: UNIX Release: xxx Version: xxx END START OS:: LINUX Release: xxx Version: xxx END START OS:: Windows Release: xxx Version: xxx ENDHere i am trying to get all the information between START and END, only if i could match OS Type. I can get all the data between the... (3 Replies)
Discussion started by: Dharmaraja
3 Replies

4. UNIX for Dummies Questions & Answers

Find the count of files by last created date based on the given date range

My unix version is IBM AIX Version 6.1 I tried google my requirement and found the below answer, find . -newermt “2012-06-15 08:13" ! -newermt “2012-06-15 18:20" But newer command is not working in AIX version 6.1 unix I have given my requirement below: Input: atr files: ... (1 Reply)
Discussion started by: yuvaa27
1 Replies

5. Shell Programming and Scripting

Date listing in a date range

Solaris 10 ksh88 Sorry for re-hashing some of this, but I can't find a proper solution in the forums. Starting with /a/archive containing (on and on date formatted directories) 20060313 20080518 20100725 20121015 20060314 20080519 ... (1 Reply)
Discussion started by: moesplace
1 Replies

6. Shell Programming and Scripting

Date and time range extraction via Awk or analysis script?

Hello does anyone know of an awk that will extract log file entries between a specific date and time range, eg: awk '/15\/Dec\/2010:16:10:00/, /15\/Dec\/2010:16:15:00/' access_log but one that works? Or a free command line log file analysis tool/script? I'd like to be able to view... (2 Replies)
Discussion started by: competitions
2 Replies

7. Shell Programming and Scripting

Separate date timestamp use awk or sed command ?

Hi, I have logfile like this : Actually the format is date format : yyyymmddHHMMSS and i want the log become this format yyyy-mm-dd HH:MM:SS for example 2009-07-19 11:46:52 Can somebody help me ? Thanks in advance (3 Replies)
Discussion started by: justbow
3 Replies

8. Shell Programming and Scripting

Get date range between 2 date input

Hi Experts, I have files name report_20090416 report_20090417 report_20090418 report_20090420 report_20090421 I have 2 input from user From Date: 20090417 To Date: 20090420 and I need to grep only those line in between. Output should be report_20090417 report_20090418... (3 Replies)
Discussion started by: tanit
3 Replies

9. Shell Programming and Scripting

How to replace a range of text with sed or awk?

Howdy! I'm trying to automate editing of a configuration file (custom.conf for GDM). I need to find every line between a line that starts with "" and the next line that starts with "", I want to preserve that line, but then delete all the lines in that configuration section and then insert... (3 Replies)
Discussion started by: TXTad
3 Replies

10. Shell Programming and Scripting

reformat date, awk and sed

The command below is getting me the output I need. awk -F"," ' { if ($6 = 475) print "@@"$3 " " "0000" $10 "0" $1 "00000000" $8}' ${DIR1}${TMPFILE1} | sed -e 's/@@1//g' > ${DIR2}${TPRFILE} Output: 900018732 00004961160200805160000000073719 Now I need to incorporate... (5 Replies)
Discussion started by: mondrar
5 Replies
Login or Register to Ask a Question