Sponsored Content
Full Discussion: awk and log files
Top Forums UNIX for Dummies Questions & Answers awk and log files Post 302516843 by Freaky on Monday 25th of April 2011 10:24:23 AM
Old 04-25-2011
awk and log files

Hello,

I have a series of logs that I need to analyse.
each looks something like:
234.10.72.175 Mon Mar 02 20:25:00 GMT 2009
226.91.87.86 Thu Mar 05 03:50:26 GMT 2009
226.91.87.86 Thu Mar 05 04:06:07 GMT 2009

Using awk, so far I have been able to count the lines in a specific file that match a patter. For example, those in Mar, 2009, between 10:00:00 and 12:00:00. But a combination of the text month and numeric year have me stumped.

an example of the lines I have been using
Code:
# between two times
awk -v "sTIME=${STARTTIME}" -v "eTIME=${ENDTIME}" '{if ($5 >= sTIME && $5 <= eTIME) print $1 }' home.hits 

#day begins with S, so a weekend
awk '$2 ~ /^S/' test.hits 

# doesnt match above...
awk '$2 !~ /^S/' test.hits

I am now having trouble with the date as they are in the mmm format.

I have thought about using sed to change the months format.
using an array and piping the output of awk into another to check.

change the times to epoch


I am currently not sure what the best way to do this is.
Cheers for looking
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Constantly updating log files (tail -f? grep? awk?)

I have a log file which is continuously added to, called log.file. I'd like to monitor this file, and when certain lines are found, update some totals in another file. I've played around with tail -f, grep, and awk, but can't seem to hit the right note, so to speak. The lines I'm... (0 Replies)
Discussion started by: nortonloaf
0 Replies

2. Shell Programming and Scripting

Merge files of differrent size with one field common in both files using awk

hi, i am facing a problem in merging two files using awk, the problem is as stated below, file1: A|B|C|D|E|F|G|H|I|1 M|N|O|P|Q|R|S|T|U|2 AA|BB|CC|DD|EE|FF|GG|HH|II|1 .... .... .... file2 : 1|Mn|op|qr (2 Replies)
Discussion started by: shashi1982
2 Replies

3. Shell Programming and Scripting

Arrange log files with AWK

Hello friends, I have too many log files to arrange. I use a simple script to create log files with below format and i forgot to create daily directory for them at the beginning. Because of this i should move all daily logs into a directory that i need to create. a part of "ls -l" output:... (1 Reply)
Discussion started by: EAGL€
1 Replies

4. Shell Programming and Scripting

Comparison and editing of files using awk.(And also a possible bug in awk for loop?)

I have two files which I would like to compare and then manipulate in a way. File1: pictures.txt 1.1 1.3 dance.txt 1.2 1.4 treehouse.txt 1.3 1.5 File2: pictures.txt 1.5 ref2313 1.4 ref2345 1.3 ref5432 1.2 ref4244 dance.txt 1.6 ref2342 1.5 ref2352 1.4 ref0695 1.3 ref5738 1.2... (1 Reply)
Discussion started by: linuxkid
1 Replies

5. Shell Programming and Scripting

Apply 'awk' to all files in a directory or individual files from a command line

Hi All, I am using the awk command to replace ',' by '\t' (tabs) in a csv file. I would like to apply this to all .csv files in a directory and create .txt files with the tabs. How would I do this in a script? I have the following script called "csvtabs": awk 'BEGIN { FS... (4 Replies)
Discussion started by: ScKaSx
4 Replies

6. Shell Programming and Scripting

How can view log messages between two time frame from /var/log/message or any type of log files

How can view log messages between two time frame from /var/log/message or any type of log files. when logfiles are very big and especially many messages with in few minutes, I would like to display log messages between 5 minute interval. Could you pls give me the command? (1 Reply)
Discussion started by: johnveslin
1 Replies

7. Shell Programming and Scripting

Comparing the matches in two files using awk when both files have their own field separators

I've two files with data like below: file1.txt: AAA,Apples,123 BBB,Bananas,124 CCC,Carrot,125 file2.txt: Store1|AAA|123|11 Store2|BBB|124|23 Store3|CCC|125|57 Store4|DDD|126|38 So,the field separator in file1.txt is a comma and in file2.txt,it is | Now,the output should be... (2 Replies)
Discussion started by: asyed
2 Replies

8. Shell Programming and Scripting

awk command to compare a file with set of files in a directory using 'awk'

Hi, I have a situation to compare one file, say file1.txt with a set of files in directory.The directory contains more than 100 files. To be more precise, the requirement is to compare the first field of file1.txt with the first field in all the files in the directory.The files in the... (10 Replies)
Discussion started by: anandek
10 Replies

9. Shell Programming and Scripting

[awk] combine and convert time from log files

dear all, an awk newbie need your help.... i have log files with this format: mylog1a.log: "08/10/2012","5:05 PM" "Hostname","Device Address","Count" "","10.10.10.18","10234" mylog2a.log: "08/11/2012","5:05 PM" "Hostname","Device Address","Count" "","10.10.10.18","12543" ... (18 Replies)
Discussion started by: makan
18 Replies

10. Shell Programming and Scripting

Redirecting log files to null writing junk into log files

Redirecting log files to null writing junk into log files. i have log files which created from below command exec <processname> >$logfile but when it reaches some size i am redirecting to null while process is running like >$logfile manually but after that it writes some junk into... (7 Replies)
Discussion started by: greenworld123
7 Replies
HTTP::Date(3)						User Contributed Perl Documentation					     HTTP::Date(3)

NAME
HTTP::Date - date conversion routines SYNOPSIS
use HTTP::Date; $string = time2str($time); # Format as GMT ASCII time $time = str2time($string); # convert ASCII date to machine time DESCRIPTION
This module provides functions that deal the date formats used by the HTTP protocol (and then some more). Only the first two functions, time2str() and str2time(), are exported by default. time2str( [$time] ) The time2str() function converts a machine time (seconds since epoch) to a string. If the function is called without an argument or with an undefined argument, it will use the current time. The string returned is in the format preferred for the HTTP protocol. This is a fixed length subset of the format defined by RFC 1123, represented in Universal Time (GMT). An example of a time stamp in this format is: Sun, 06 Nov 1994 08:49:37 GMT str2time( $str [, $zone] ) The str2time() function converts a string to machine time. It returns "undef" if the format of $str is unrecognized, otherwise whatever the "Time::Local" functions can make out of the parsed time. Dates before the system's epoch may not work on all operating systems. The time formats recognized are the same as for parse_date(). The function also takes an optional second argument that specifies the default time zone to use when converting the date. This parameter is ignored if the zone is found in the date string itself. If this parameter is missing, and the date string format does not contain any zone specification, then the local time zone is assumed. If the zone is not ""GMT"" or numerical (like ""-0800"" or "+0100"), then the "Time::Zone" module must be installed in order to get the date recognized. parse_date( $str ) This function will try to parse a date string, and then return it as a list of numerical values followed by a (possible undefined) time zone specifier; ($year, $month, $day, $hour, $min, $sec, $tz). The $year will be the full 4-digit year, and $month numbers start with 1 (for January). In scalar context the numbers are interpolated in a string of the "YYYY-MM-DD hh:mm:ss TZ"-format and returned. If the date is unrecognized, then the empty list is returned ("undef" in scalar context). The function is able to parse the following formats: "Wed, 09 Feb 1994 22:23:32 GMT" -- HTTP format "Thu Feb 3 17:03:55 GMT 1994" -- ctime(3) format "Thu Feb 3 00:00:00 1994", -- ANSI C asctime() format "Tuesday, 08-Feb-94 14:15:29 GMT" -- old rfc850 HTTP format "Tuesday, 08-Feb-1994 14:15:29 GMT" -- broken rfc850 HTTP format "03/Feb/1994:17:03:55 -0700" -- common logfile format "09 Feb 1994 22:23:32 GMT" -- HTTP format (no weekday) "08-Feb-94 14:15:29 GMT" -- rfc850 format (no weekday) "08-Feb-1994 14:15:29 GMT" -- broken rfc850 format (no weekday) "1994-02-03 14:15:29 -0100" -- ISO 8601 format "1994-02-03 14:15:29" -- zone is optional "1994-02-03" -- only date "1994-02-03T14:15:29" -- Use T as separator "19940203T141529Z" -- ISO 8601 compact format "19940203" -- only date "08-Feb-94" -- old rfc850 HTTP format (no weekday, no time) "08-Feb-1994" -- broken rfc850 HTTP format (no weekday, no time) "09 Feb 1994" -- proposed new HTTP format (no weekday, no time) "03/Feb/1994" -- common logfile format (no time, no offset) "Feb 3 1994" -- Unix 'ls -l' format "Feb 3 17:03" -- Unix 'ls -l' format "11-15-96 03:52PM" -- Windows 'dir' format The parser ignores leading and trailing whitespace. It also allow the seconds to be missing and the month to be numerical in most formats. If the year is missing, then we assume that the date is the first matching date before current month. If the year is given with only 2 digits, then parse_date() will select the century that makes the year closest to the current date. time2iso( [$time] ) Same as time2str(), but returns a "YYYY-MM-DD hh:mm:ss"-formatted string representing time in the local time zone. time2isoz( [$time] ) Same as time2str(), but returns a "YYYY-MM-DD hh:mm:ssZ"-formatted string representing Universal Time. SEE ALSO
"time" in perlfunc, Time::Zone COPYRIGHT
Copyright 1995-1999, Gisle Aas This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. perl v5.18.2 2012-03-30 HTTP::Date(3)
All times are GMT -4. The time now is 12:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy