Sponsored Content
Top Forums Shell Programming and Scripting Extracting log entries from a date onwards Post 302511680 by rijeshpp on Thursday 7th of April 2011 10:17:35 AM
Old 04-07-2011
Extracting log entries from a date onwards

One of the log file looks like entries as below.

Code:
Wed Apr  6 14:51:18 2011 [pid 24366] [wasadmin] FAIL LOGIN: Client "9.191.21.54"
Wed Apr  6 14:52:53 2011 [pid 25554] CONNECT: Client "9.191.21.54"
Wed Apr  6 14:52:54 2011 [pid 25553] [wasadmin] OK LOGIN: Client "9.191.21.54"
Wed Apr  6 14:55:10 2011 [pid 27442] CONNECT: Client "9.191.21.54"
Wed Apr  6 14:55:12 2011 [pid 27441] [wasadmin] FAIL LOGIN: Client "9.191.21.54"
Wed Apr  6 14:56:12 2011 [pid 28291] CONNECT: Client "9.191.21.54"
Wed Apr  6 14:56:13 2011 [pid 28290] [wasadmin] OK LOGIN: Client "9.191.21.54"
Wed Apr  6 15:11:57 2011 [pid 8774] CONNECT: Client "9.191.21.54"
Wed Apr  6 15:11:57 2011 [pid 8773] [wasadmin] OK LOGIN: Client "9.191.21.54"
Thu Apr  7 19:02:54 2011 [pid 18139] CONNECT: Client "9.122.71.33"
Thu Apr  7 19:03:00 2011 [pid 18138] [wasadmin] FAIL LOGIN: Client "9.122.71.33"
Thu Apr  7 19:03:09 2011 [pid 18146] CONNECT: Client "9.122.71.33"
Thu Apr  7 19:03:15 2011 [pid 18145] [rijesh] OK LOGIN: Client "9.122.71.33"
Thu Apr  7 19:03:21 2011 [pid 18484] CONNECT: Client "9.122.71.33"
Thu Apr  7 19:03:24 2011 [pid 18483] [wasadmin] OK LOGIN: Client "9.122.71.33"
Thu Apr  7 19:03:29 2011 [pid 18492] CONNECT: Client "9.122.71.33"
Thu Apr  7 19:03:35 2011 [pid 18491] [wasadmin] FAIL LOGIN: Client "9.122.71.33"

From the above file, I want to develop a shell script that extract all lines lines from the above log file from the position onwards last time it executed(The last execution time will be recorded). For instance the log file may have thousands of line; the intention is to run the shell script as cronjob with a frequency of 15 minutes. So when the first time cronjob runs it will read all the files and the subsequent execution it needs to extract the lines of log entries from and after the time of script execution.

Having said the above requirement; I am looking for one piece of information here in this forum. I can get the execution time of the script in time format lets say using "date '+%T'". For example if I am executing script (cronjob) in the time of 19:00:00; then extract all the lines using grep that has entries after logged in 19:00:00 .I can have a logic of reading the log file line by line by incorporating a logic of incrementing execution time. However I suspect that may make the operation expensive if the log file has too many entries. So I am looking for a logic that is competitively less expensive operation. Any advice in this regards would be great help.

Thanks in advance,
Rijesh.

Last edited by vbe; 04-07-2011 at 11:20 AM.. Reason: typos: missng end to decode tag
 

10 More Discussions You Might Find Interesting

1. Shell Programming and Scripting

Perl: Extracting date from file name and comparing with current date

I need to extract the date part from the file name (20080221 in this ex) and compare it with the current date and delete it, if it is a past date. $file = exp_ABCD4_T-2584780_upto_20080221.dmp.Z really appreciate any help. thanks mkneni (4 Replies)
Discussion started by: MKNENI
4 Replies

2. UNIX for Dummies Questions & Answers

extracting date from a filename

Hi, I am a beginner in Unix so please bear with me... I have a directory which has files in format: RECF-YYYY-MM-DD-input. For example, RECF-2008-02-25-input. I need to extract the YYYYY-MM-DD substring from this filename and convert that into date and compare it with a date. How do I do that?... (7 Replies)
Discussion started by: laiko
7 Replies

3. Shell Programming and Scripting

Extracting Date from string

Hi Gurus I want to extract a date and version code which shall come in filename consisting of underscores. The filename can contain any / one underscores but the version number will come after date and will be separted by underscore String formats ============= ABC_20090815_2.csv... (13 Replies)
Discussion started by: r_t_1601
13 Replies

4. Shell Programming and Scripting

Extracting data from two date entries

Hi again: I have this file: "2010-11-1 11:50:00",40894,13.38,17.24,12.92,13.23,"2010-11-14 11:43:02",12.56,"2010-11-14 11:46:02",22.68,20.95,"2010-11-14 11:44:03",2.144,2.078,190.4,14.27,6.293,"2010-11-14 ... (2 Replies)
Discussion started by: iga3725
2 Replies

5. Homework & Coursework Questions

extracting date from log file

You are given a 1 year logfile with each line starting with a date in the form “YYYY-MM-DD”. How would you extract logs from the 4th day of each month and put them into a new file (1 Reply)
Discussion started by: DOkuwa
1 Replies

6. Shell Programming and Scripting

Extracting data from a log file with date formats

Hello, I have a log file for the year, which contains lines starting with the data in the format of YYYY-MM-DD. I need to get all the lines that contain the DD being 04, how would I do this? I tried using grep "*-*04" but it didn't work. Any quick one liners I should know about? Thank you. (2 Replies)
Discussion started by: cpickering
2 Replies

7. Shell Programming and Scripting

Need to check a file from a certain position and date onwards

Hi Guys, I need some advice please. My script is not grabbing information from a text file from a certain date correctly. It seems to be grabbing everying in the file, i know it is something simple but i have looked to hard and to long, to know what the issue is. Script awk '... (9 Replies)
Discussion started by: Junes
9 Replies

8. Shell Programming and Scripting

Delete log files content older than 30 days and append the lastest date log file date

To delete log files content older than 30 days and append the lastest date log file date in the respective logs I want to write a shell script that deletes all log files content older than 30 days and append the lastest log file date in the respective logs This is my script cd... (2 Replies)
Discussion started by: sreekumarhari
2 Replies

9. Shell Programming and Scripting

Monitor log entries in log files with no Date format? - Efficient logcheck?

is there a way to efficiently monitor logfiles that do not have a date or time format? i have several logs on several different servers that need to be monitored. but i realized writing a script for this would be very complex and time consuming giving the variety of things i need to check for i.e.... (2 Replies)
Discussion started by: SkySmart
2 Replies

10. Shell Programming and Scripting

Extracting log files based on date and time.

Hi All, i have some log files generated in a folder daily with the format abc.def.20130306.100001 ghi.jkl.20130306.100203 abc.def.20130305.100001 ghi.jkl.20130305.100203 the format is the date followed by time . all i want is to get the files that are generated for todays... (3 Replies)
Discussion started by: mahesh300182
3 Replies
sched_getparam(3C)					   Standard C Library Functions 					sched_getparam(3C)

NAME
sched_getparam - get scheduling parameters SYNOPSIS
#include <sched.h> int sched_getparam(pid_t pid, struct sched_param *param); DESCRIPTION
The sched_getparam() function returns the scheduling parameters of a process specified by pid in the sched_param structure pointed to by param. The only required member of param is sched_priority. If a process specified by pid exists and if the calling process has permission, the scheduling parameters for the process whose process ID is equal to pid will be returned. If pid is 0, the scheduling parameters for the calling process will be returned. The behavior of the sched_getparam() function is unspeci- fied if the value of pid is negative. RETURN VALUES
Upon successful completion, the sched_getparam() function returns 0. If the call to sched_getparam() is unsuccessful, the function returns -1 and sets errno to indicate the error. ERRORS
The sched_getparam() function will fail if: EPERM The requesting process does not have permission to obtain the scheduling parameters of the specified process. ESRCH No process can be found corresponding to that specified by pid. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ |ATTRIBUTE TYPE |ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Interface Stability |Committed | +-----------------------------+-----------------------------+ |MT-Level |MT-Safe | +-----------------------------+-----------------------------+ |Standard |See standards(5). | +-----------------------------+-----------------------------+ SEE ALSO
librt(3LIB), sched.h(3HEAD), sched_getscheduler(3C), sched_setparam(3C), sched_setscheduler(3C), attributes(5), standards(5) SunOS 5.11 1 Apr 2008 sched_getparam(3C)
All times are GMT -4. The time now is 04:15 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy