Sponsored Content
Top Forums Shell Programming and Scripting Suggestions for Inserting Missing Timestamps Post 302395060 by st33lz on Sunday 14th of February 2010 02:55:27 PM
Old 02-14-2010
Suggestions for Inserting Missing Timestamps

I have apache logs that I'm trying to create some metrics with. I've stripped the files down to number of request per second. Example:

03:04:01 3 (hour:minute:second #of requests)
03:04:03 2
03:04:04 1
03:04:05 1
03:04:07 2

My problem is that not every second has a request so of course no log entry for that given second. I need to insert each missing second and a zero for the request.

I'm really not sure how to go about doing that. I guess create a loop that would create a time stamp for each second of the 24 hour period (84600 seconds) and then grep the file for that second. If the second does not exist then cat the line to the file, and eventually sort the file. Seems like that would take up a pretty big chunk of time.

Creating a loop to generate each time stamp is beyond my scripting skills and anything I've tried has come up way short.

Anyone have a solution using BASH, SH, PERL...

Thanks for any suggestions.
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

Unix timestamps

Can someone help me with a Unix or perl script to convert the unix timestamps to human readable format? Any help will be highly appreciated... (3 Replies)
Discussion started by: hamsasal
3 Replies

2. Solaris

Difference between two timestamps

I'm writting a script to find the difference between two timestamp. One field i get on delivery time of the file like 07:17 AM and other is my SLA time 06:30 AM I need to find the difference between these two time (time exceeded to meet SLA). Need some suggestions. (8 Replies)
Discussion started by: raman1605
8 Replies

3. Shell Programming and Scripting

timestamps

Hello! I have the following problem. I read a file using perl, each line of this file has the fllowing format. 14/4/2008 8:42:03 πμ|10800|306973223399|4917622951117|1||1259|1|126|492|433||19774859454$ Th first field is the timestamp and the second field is the offset in seconds. How can... (1 Reply)
Discussion started by: chriss_58
1 Replies

4. Shell Programming and Scripting

Compare Timestamps

Hi! Long time reader first time registered user and poster. I've picked up some times and tricks and I'm at a dead end... I've parsed a log file for duplicates and printed only the two fields I need (duplicate entry and time stamp). My question is, with this output, how would I script... (2 Replies)
Discussion started by: rexpokinghorn
2 Replies

5. Shell Programming and Scripting

Help with touch and timestamps

Hello fellow Unix geeks, I have been given a very urgent assignment in my office on writing a particular Shell script but I'm very much new to it.I would appreciate any help from you on solving this problem--which might seem very trivial to you. The Unix flavour is a Sun Solaris one..(not... (6 Replies)
Discussion started by: Digjoy83
6 Replies

6. Shell Programming and Scripting

Comparing two timestamps

Hi all!!, I'm using Ksh and working on Linux. I want to compare two timestamps, timestamp1 and timestamp2. Until, timestamp1 is lesser than timestamp2, i want to do something, lets say print something. The code i have written is: a=`date +%H:%M:%S` b=`date +%H:%M:%S -d" 1... (1 Reply)
Discussion started by: Jayaraman
1 Replies

7. SuSE

How to resolve missing missing dependencies with opensuse 11.3 and 12.3?

Hello, This is a programming question as well as a suse question, so let me know if you think I should post this in programming. I have an application that I compiled under opensuse 12.2 using g77-3.3/g++3.3. The program compiles and runs just fine. I gave the application to a colleague who... (2 Replies)
Discussion started by: LMHmedchem
2 Replies

8. Red Hat

Yum - resolving missing dependencies that are not missing

I am trying to install VirtualBox on RHEL 5 but I need the 32 bit version for 32 bit Windows. When I run yum I get the following: sudo yum localinstall /auto/spvtg-it/spvss-migration/Software/VirtualBox-4.3-4.3.2_90405_el6-1.i686.rpm Loaded plugins: fastestmirror Setting up Local Package... (13 Replies)
Discussion started by: gw1500se
13 Replies

9. Shell Programming and Scripting

Write with a look for timestamps

hello i'm using SOX to generate a spectrogram from a wave file with the command : #sox file.wav -n spectrogram is there a way to create a spectrogram using the same command but reading file timestamps instead of the namefile.wav , since name is changing every 4 hours? (it's saved with... (2 Replies)
Discussion started by: Board27
2 Replies

10. Shell Programming and Scripting

Merging multiple lines to columns with awk, while inserting commas for missing lines

Hello all, I have a large csv file where there are four types of rows I need to merge into one row per person, where there is a column for each possible code / type of row, even if that code/row isn't there for that person. In the csv, a person may be listed from one to four times... (9 Replies)
Discussion started by: RalphNY
9 Replies
ROTATELOGS(8)							    rotatelogs							     ROTATELOGS(8)

NAME
rotatelogs - Piped logging program to rotate Apache logs SYNOPSIS
rotatelogs [ -l ] [ -f ] logfile rotationtime|filesizeM [ offset ] SUMMARY
rotatelogs is a simple program for use in conjunction with Apache's piped logfile feature. It supports rotation based on a time interval or maximum size of the log. OPTIONS
-l Causes the use of local time rather than GMT as the base for the interval or for strftime(3) formatting with size-based rotation. Note that using -l in an environment which changes the GMT offset (such as for BST or DST) can lead to unpredictable results! -f Causes the logfile to be opened immediately, as soon as rotatelogs starts, instead of waiting for the first logfile entry to be read (for non-busy sites, there may be a substantial delay between when the server is started and when the first request is handled, meaning that the associated logfile does not "exist" until then, which causes problems from some automated logging tools). Available in version 2.2.9 and later. logfile The path plus basename of the logfile. If logfile includes any '%' characters, it is treated as a format string for strftime(3). Otherwise, the suffix .nnnnnnnnnn is automatically added and is the time in seconds. Both formats compute the start time from the beginning of the current period. For example, if a rotation time of 86400 is specified, the hour, minute, and second fields created from the strftime(3) format will all be zero, referring to the beginning of the current 24-hour period (midnight). rotationtime The time between log file rotations in seconds. The rotation occurs at the beginning of this interval. For example, if the rotation time is 3600, the log file will be rotated at the beginning of every hour; if the rotation time is 86400, the log file will be rotated every night at midnight. (If no data is logged during an interval, no file will be created.) filesizeM The maximum file size in megabytes followed by the letter M to specify size rather than time. offset The number of minutes offset from UTC. If omitted, zero is assumed and UTC is used. For example, to use local time in the zone UTC -5 hours, specify a value of -300 for this argument. In most cases, -l should be used instead of specifying an offset. EXAMPLES
CustomLog "|bin/rotatelogs /var/logs/logfile 86400" common This creates the files /var/logs/logfile.nnnn where nnnn is the system time at which the log nominally starts (this time will always be a multiple of the rotation time, so you can synchronize cron scripts with it). At the end of each rotation time (here after 24 hours) a new log is started. CustomLog "|bin/rotatelogs -l /var/logs/logfile.%Y.%m.%d 86400" common This creates the files /var/logs/logfile.yyyy.mm.dd where yyyy is the year, mm is the month, and dd is the day of the month. Logging will switch to a new file every day at midnight, local time. CustomLog "|bin/rotatelogs /var/logs/logfile 5M" common This configuration will rotate the logfile whenever it reaches a size of 5 megabytes. ErrorLog "|bin/rotatelogs /var/logs/errorlog.%Y-%m-%d-%H_%M_%S 5M" This configuration will rotate the error logfile whenever it reaches a size of 5 megabytes, and the suffix to the logfile name will be cre- ated of the form errorlog.YYYY-mm-dd-HH_MM_SS. PORTABILITY
The following logfile format string substitutions should be supported by all strftime(3) implementations, see the strftime(3) man page for library-specific extensions. o %A - full weekday name (localized) o %a - 3-character weekday name (localized) o %B - full month name (localized) o %b - 3-character month name (localized) o %c - date and time (localized) o %d - 2-digit day of month o %H - 2-digit hour (24 hour clock) o %I - 2-digit hour (12 hour clock) o %j - 3-digit day of year o %M - 2-digit minute o %m - 2-digit month o %p - am/pm of 12 hour clock (localized) o %S - 2-digit second o %U - 2-digit week of year (Sunday first day of week) o %W - 2-digit week of year (Monday first day of week) o %w - 1-digit weekday (Sunday first day of week) o %X - time (localized) o %x - date (localized) o %Y - 4-digit year o %y - 2-digit year o %Z - time zone name o %% - literal `%' Apache HTTP Server 2008-05-10 ROTATELOGS(8)
All times are GMT -4. The time now is 10:04 PM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy