Sponsored Content
Top Forums Shell Programming and Scripting Processing a log file based on date/time input and the date/time on the log file Post 302175741 by primp on Saturday 15th of March 2008 04:21:30 PM
Old 03-15-2008
Processing a log file based on date/time input and the date/time on the log file

Hi,

I'm trying to accomplish the following and would like some suggestions or possible bash script examples that may work

I have a directory that has a list of log files that's periodically dumped from a script that is crontab that are rotated 4 generations. There will be a time stamp that is associated when the file is created.

application.log.1
application.log.2
application.log.3
application.log.4
application.log.5

I have another script that will take an input of date/time, I'm not sure which is the easiest format to compare, from the short look around it sounds like epoch is the easiest/fastest.

I would like to take that format whichever it is, and basically subtract 5minutes which is a poll period thats been defined and then use that date/time to compare with the logs that are rotated and basically process the log that closely matches the "currentInputDateTime" where it can not be larger than "currentInputDateTime" but less than/equal to the current time, I assume this might be a "ls -tr | awk '{print $6 $7}' to get the date/time and somehow convert those to epoch.

I was wondering if this was something that has been encountered before so the wheel does not have to be re-invented.

Thanks in advance
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Inserting Date&Time Stamp In Existing Log File

I am trying to insert a line with a date stamp in a file that is used to monitor activity in one of our directories. By doing this, I want to grep that file each day and go to the last entry for each time a error occurred and pull all errors generated if any exist. If error exists I want that error... (3 Replies)
Discussion started by: shephardfamily
3 Replies

2. Shell Programming and Scripting

Date and time log file

Hi, I wrote a small perl script in unix that searches in a file and saves some information in a separate file. Since this is a log file, I would like to have the date added to file name. I have no idea where to start. output: log_010907.txt thanks ken (8 Replies)
Discussion started by: captoro
8 Replies

3. UNIX for Dummies Questions & Answers

Adding date and time to a log file

Morning all Im hoping you can help me. We have a nice new oracle server :( and are needing to move some files around for EDI and BACS. The server runs windows but has an app called MKS toolkit installed which give unix commands. (Needed for the oracle stuff) I have had a go using dos commands... (2 Replies)
Discussion started by: ltodd2
2 Replies

4. Shell Programming and Scripting

Extract info from log file and compute using time date stamp

Looking for a shell script or a simple perl script . I am new to scripting and not very good at it . I have 2 directories . One of them holds a text file with list of files in it and the second one is a daily log which shows the file completion time. I need to co-relate both and make a report. ... (0 Replies)
Discussion started by: breez_drew
0 Replies

5. Shell Programming and Scripting

Delete log file entries based on the Date/Timestamp within log file

If a log file is in the following format 28-Jul-10 ::: Log message 28-Jul-10 ::: Log message 29-Jul-10 ::: Log message 30-Jul-10 ::: Log message 31-Jul-10 ::: Log message 31-Jul-10 ::: Log message 1-Aug-10 ::: Log message 1-Aug-10 ::: Log message 2-Aug-10 ::: Log message 2-Aug-10 :::... (3 Replies)
Discussion started by: vikram3.r
3 Replies

6. Shell Programming and Scripting

Grep the Content of a LOG File which has latest Date and Time

Hi All, Need a small help. I have a log file which keeps updating for every Minute with multiple number of lines. I just want to grep few properties which has latest Date and Time to it. How do i do it? I wanted to grep a property by name "Reloading cache with a maximum of" from the... (4 Replies)
Discussion started by: nvindraneel
4 Replies

7. Shell Programming and Scripting

Delete log files content older than 30 days and append the lastest date log file date

To delete log files content older than 30 days and append the lastest date log file date in the respective logs I want to write a shell script that deletes all log files content older than 30 days and append the lastest log file date in the respective logs This is my script cd... (2 Replies)
Discussion started by: sreekumarhari
2 Replies

8. Shell Programming and Scripting

Extracting log files based on date and time.

Hi All, i have some log files generated in a folder daily with the format abc.def.20130306.100001 ghi.jkl.20130306.100203 abc.def.20130305.100001 ghi.jkl.20130305.100203 the format is the date followed by time . all i want is to get the files that are generated for todays... (3 Replies)
Discussion started by: mahesh300182
3 Replies

9. Shell Programming and Scripting

Search for logs traced between specific date and time from log file

HI, I want to search for a logs which are trace between specific date and time from logs file. My logs are generated like this :- Tue Jun 18 05:00:02 EEST 2013 | file_check.sh| Message:script has files to process. Thu Jun 20 05:00:02 EEST 2013 | file_check.sh| Message:script has files to... (5 Replies)
Discussion started by: ketanraut
5 Replies

10. Shell Programming and Scripting

Shell Script | Parse log file after a given date and time stamp

I am developing one script which will take log file name, output file name, date, hour and minute as an argument and based on these inputs, the script will scan and capture all the error(s) that have been triggered from a given time. Example: script should capture all the error after 13:50 on Jan... (2 Replies)
Discussion started by: ROMA3
2 Replies
ncab2clf(1)							   User Commands						       ncab2clf(1)

NAME
ncab2clf - convert binary log file to Common Log File format SYNOPSIS
/usr/bin/ncab2clf [-Dhv] [-i input-file] [-o output-file] [-b size] [-n number] [-s datetime] DESCRIPTION
The ncab2clf command is used to convert the log file generated by the Solaris Network Cache and Accelerator ("NCA") from binary format, to Common Log File ("CLF") format. If no input-file is specified, ncab2clf uses stdin. If no output-file is specified, the output goes to std- out. OPTIONS
-b Specifies the binary-log-file blocking in kilobytes; the default is 64 Kbyte. -D Specifies that direct I/O be disabled. -h Prints usage message. -i input-file Specifies the input file. -n number Output number CLF records. -o output-file Specifies the output file. -s datetime Skip any records before the date and time specified in datetime. You can specify the date and time in CLF format or in the format specified by the touch(1) utility. CLF format is the dominant format, so ncab2clf first analyzes datetime assuming CLF. -v Provides verbose output. EXAMPLES
Example 1: Converting a Binary File to a Common Log File Format The following example converts the binary file /var/nca/logs/nca.blf to a file /var/nca/logs/nca.clf, which is in Common Log File format. example% ncab2clf -D -i /var/nca/logs/nca.blf -o /var/nca/logs/nca.clf Example 2: Converting Multiple Log Files The following script may be used to convert multiple log files. The directory designated by "*" must only contain log files. !/bin/ksh for filename in * do ncab2clf -D < $filename > $filename.clf done Example 3: Using -s and -n on a Raw Device The following example shows how ncab2clf can be used on a raw device. If not using the -n option, the default is to convert all records from the starting location to the end of the file. The date and time specified with -s, below, is in CLF format. example% ncab2clf -s '10/Apr/2001:09:23:13' -n 100 < /dev/dsk/c2t1d0s6 EXIT STATUS
The following exit values are returned: 0 The file converted successfully >0 An error occurred. ATTRIBUTES
See attributes(5) for descriptions of the following attributes: +-----------------------------+-----------------------------+ | ATTRIBUTE TYPE | ATTRIBUTE VALUE | +-----------------------------+-----------------------------+ |Availability |SUNWncau | +-----------------------------+-----------------------------+ |Interface Stability |Evolving | +-----------------------------+-----------------------------+ SEE ALSO
nca(1), ncakmod(1), nca.if(4), ncakmod.conf(4), ncalogd.conf(4), attributes(5) System Administration Guide: IP Services NOTES
The binary log files generated by NCA can become very large. When converting these large binary files, use the -b option to the ncab2clf command to help performance. Direct I/O is a benefit to the user if the data being written does not come in as large chunks. However, if the user wishes to convert the log file in large chunks using the -b option, then direct I/O should be disabled by using the -D option. SunOS 5.10 28 Sep 2001 ncab2clf(1)
All times are GMT -4. The time now is 11:30 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy