Parsing log files, displaying logs between specific dates


 
Thread Tools Search this Thread
Top Forums Shell Programming and Scripting Parsing log files, displaying logs between specific dates
# 1  
Old 03-05-2009
Parsing log files, displaying logs between specific dates

Sorry, couldn't really think of a simple subject/title.

So, I have a log file, and the dates are displayed like so:

2009-03-05 02:49:44

So the first and second field are the date/time. I can change them into a unix timestamp easily with:

date -d "2009-03-05 02:49:44" +%s

However, trying to do that is not as easy as I thought. I wanted to get all logs between X and Y time (lets say for a few hours on Z day). To make this easily reusable, I would rather do it this way.

I have tried to use awk with the system() function, but maybe I did not use it correctly..

awk '{ system("date -d " $1,$2 " +%s"); }'

Of course, I have tried other varieties of this, but to no avail. I have thought of possibly doing it other ways, but I am not even sure what will work now.. Anyone have any suggestions?
# 2  
Old 03-05-2009
assume X and Y are already variables that are in epoch seconds ie +%s format.
Code:
while read t1 t2 restofline
do
      t=$(date -d "$t1 $t2" +%s)
      if [[ $(( $t >= $X )) -eq 1  &&  $(( $t <= $Y )) -eq 1 ]] ; then
      	echo "$t1 $t2 $restofline"
      fi
done < logfile > resultfile

# 3  
Old 03-05-2009
can anybody me on this its urgent


i have a script

echo "enter date:\c"
read date

if [ -z "$date" ]
then
echo "Please enter the date"
fi


now when i run the script and enter the date in say

05032009

then i want to print o/p as

05 March 2009

how do i do this

thanks,
haris
# 4  
Old 03-05-2009
With gawk time and string functions:
input file:
Code:
127.0.0.1 - - 2009-03-05 17:17:25 "GET / HTTP/1.1" 304 - "-"

Code:
awk 'mktime(gensub(/-|:/, " ", "g", $4" "$5)) > 1236294000 ' file

This will print lines with date > 2009-03-06. Just add a condition for the lower boundary and you are done.

The gensub() string function replaces the - and : characters by space as the mktime() function only accepts the YYYY MM DD HH MM SS format.

Check if your awk version supports these functions.
# 5  
Old 03-06-2009
is there any other way to get
05032009
as
05 March 2009

bcoz i didnt got the above code exactly

thanks
haris
# 6  
Old 03-06-2009
@Haris, this is another problem than the OP. But however, here is a possible solution depending on your distribution. This one is using GNU coreutils date command.

Code:
dte=05032009
dteIso=$(echo $dte | sed 's/\(..\)\(..\)\(....\)/\3\2\1/')
LC_ALL=C date -d $dteIso +'%d %B %Y'

# 7  
Old 03-06-2009
hi i have tried your previous code but it didnt wrk
thats the mean reason for previous post

so could u please help me with this
i just want to know how to convert

figure to text

if u can please explain me with example
as i m quite new to unix scripting


thanks
haris
Login or Register to Ask a Question

Previous Thread | Next Thread

10 More Discussions You Might Find Interesting

1. Red Hat

Moving files with specific dates

Hi, These are the list of files in one directory in the server : # ls -lrt total 10120 -rw-r--r-- 1 root root 4484 Jul 8 2011 install.log.syslog -rw-r--r-- 1 root root 51890 Jul 8 2011 install.log -rw------- 1 root root 3140 Jul 8 2011 anaconda-ks.cfg drwxr-xr-x 2 root root... (2 Replies)
Discussion started by: anaigini45
2 Replies

2. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

3. Shell Programming and Scripting

Redirect all logs files contents into a single log file

Hi , I have a Data cleansing process which creates different log file for each step , when the process runs it creates following log files in below order: p1_tranfrmr_log.txt p1_tranfrmr_stats.txt p2_globrtr_log.txt p2_globrtr_stats.txt p3_cusparse_log.txt p3_cusparse_stats.txt ' '... (8 Replies)
Discussion started by: sonu_pal
8 Replies

4. Debian

Logrotate truncated my log files to 0 bytes and no logs are written

Hi, Yesterday I installed and configured logrotate on my Debian machine. I was expecting this to run at 06:25 in the morning and it actually did. All my old logs were compressed and zipped but the new logs were all with size equal to 0 bytes. Processes, while still running ok, they were not... (2 Replies)
Discussion started by: pmatsinopoulos
2 Replies

5. UNIX and Linux Applications

Parsing Tuxedo Logs

Right now I am parsing Tuxedo logs to calculate response times for various services. I was hoping to find a log tool that had support for Tuxedo and would generate drill down html reports. ---------- Post updated at 02:35 PM ---------- Previous update was at 02:33 PM ---------- I just wanted... (0 Replies)
Discussion started by: Lurch
0 Replies

6. Shell Programming and Scripting

Help parsing logs maybe with menu and variables?

I would like to parse through some logs looking for things like exception or failed (grep -i failed). Ideal would be if it were in a menu format so someone without unix ability could just choose option 1 2 or 3 etc. If I could pass the hostname to a variable also that would be awesome, so someone... (5 Replies)
Discussion started by: taekwondo
5 Replies

7. UNIX for Dummies Questions & Answers

To find files with specific dates and cp another folder.

Hi all, We have an existing script: find /u03/oraprod/perpcomn/admin/out -type f -ctime +7 \ -exec cp {} "/u08/oraprod/backup/cout" \; Which is to find all files more than 7 days and copy to another folder. However I would like to only list files with Sep 29, and cp to another folder. ... (2 Replies)
Discussion started by: *Jess*
2 Replies

8. Shell Programming and Scripting

Parsing out the logs and generating report

My file will contain following(log.txt): start testcase: config loading ...... error XXXX ..... end testcase: config loading, result failed start testcase: ping check ..... error ZZZZZ ..... error AAAAA end testcase: Ping check, result failed I am expecting below output. ... (4 Replies)
Discussion started by: shellscripter
4 Replies

9. Shell Programming and Scripting

Getting list of all the log files between two dates

I need to get the list of all the log files for a particular duration, say between two dates,i.e I need to get the list of all the log files between date1 and date2.The two dates are entered by the user. The format of the log file is : /path_name/graph_name_20080801180308.log. I dont... (1 Reply)
Discussion started by: avishekp
1 Replies

10. Shell Programming and Scripting

reading files for specific dates

assume files are in a directory /data $ ls -ltr Just displaying Data and file name: 01/01/2004 file_3434_typea.dat 01/01/2004 file_3423_typea.dat 01/01/2004 file_3436_typea.dat 01/01/2004 file_3434_typeb.dat 01/01/2004 file_3423_typeb.dat 01/01/2004 file_3436_typeb.dat ... (3 Replies)
Discussion started by: siva_jm
3 Replies
Login or Register to Ask a Question