Sponsored Content
Top Forums Shell Programming and Scripting Need Time Stamp Range On Log Files Post 302948721 by Don Cragun on Wednesday 1st of July 2015 05:20:33 PM
Old 07-01-2015
What operating system and shell are you using? (The tools available to process timestamps vary considerably from OS to OS, and in some cases from shell to shell.)

Are your log files sorted in increasing timestamp order?

Will the timestamp range you want to select ever cross day boundaries or will the start and end points always be on the same date?

Can you specify the timestamp format that the user has to enter for the start and end points (e.g., YYYYMMDDhhmmss, YYYYMMDD:hh:mm:ss, or DD/Mon/YYYY:hh:mm:ss)? If so, have you chosen a format, or can we select one that makes processing easier? If not, what format(s) do(es) your script need to be able to interpret?

Will all of the offsets from GMT (i.e., -0400) be the same for the timestamp ranges you're interested in? (Your script will be much more complex if log entries are coming from various timezones around the world or if your script has to make adjustments for timestamp ranges crossing shifts to and from daylight savings time.)

Is the output supposed to change at all (other than reducing the number of selected entires being processed)? If so, how?
 

10 More Discussions You Might Find Interesting

1. UNIX for Dummies Questions & Answers

How to search for files based on the time stamp

Hi All, I know the timestamp of a file. Now i would like to list all the files in the with the same time stamp in the same file. Any help would be appreciated. Thanks. sunny (1 Reply)
Discussion started by: sunny_03
1 Replies

2. Solaris

doubt reg time stamp in files.

I copied a file from one host to another using sftp. But after copying the time stamp is not updating . Even though I checked the permission, it looks good. I copied the same file to some temporary location, there it updating the time stamp. Anyone have any idea on this (6 Replies)
Discussion started by: rogerben
6 Replies

3. Shell Programming and Scripting

time stamp perl script error out of range 1..31

Hi, while running the perl script i am getting this error message , Day '' out of range 1..31 at rsty.sh line 44 what do iam missing in the script, any suggestion #!/usr/bin/perl use Time::Local; my $wday = $ARGV; my $month = $ARGV; # convert the month shortname into 0-11 number if... (4 Replies)
Discussion started by: saha
4 Replies

4. Shell Programming and Scripting

Old time stamp being updated for new files

Hello Friends I am facing a weird problem :confused:, we receive thousands of files in my system on a daily basis, access time stamp on some of the files are being updated as old time stamp like 1968-01-19, Could some one help me what could be causing this? so that i can narrow down the problem... (4 Replies)
Discussion started by: Prateek007
4 Replies

5. Shell Programming and Scripting

Identify log files based on time stamp,zip and then copy..HELP

Hi All, PFB is a requirement. I am new to shell scripting. So plz help. It would be highly appreciated. 1. choose all the log files based on a particular date (files location is '/test/domain')--i.e,we should choose all the files that are modified on 29th November, neither 28th nor 30th 2.... (3 Replies)
Discussion started by: skdas_niladri
3 Replies

6. Shell Programming and Scripting

Select files by time stamp

Hi, I need help to read file in a directory on basis of time stamp. e.g. If file access in last 2 minutes it should not be copy to remote directory. Below is my script. +++++++++++++++++++++++++ #!/bin/ksh DATE=`date +"%Y-%m-%d_%H%M"` SEPARATER=" " exec < out_interfaces.cfg... (1 Reply)
Discussion started by: qamar.alam
1 Replies

7. Shell Programming and Scripting

awk - check time stamp between range or not

I want to check given time stamp is between the given time stamp or not. I am using AIX. YYYYMMDDHHMMSS abc.csv START TIME, END TIME 20130209018000,20130509022000 20120209018000,20130509022000 20120209018000,20130509022000 Script will check given time stamp is between above two range or... (2 Replies)
Discussion started by: vegasluxor
2 Replies

8. Shell Programming and Scripting

Files with date and time stamp

Hi Folks, Need a clarification on files with date and time stamp. Here is my requirement. There is a file created everyday with the following format "file.txt.YYYYMMDDHHMMSS". Now i need to check for this file and if it is available then i need to do some task to the file. I tried... (6 Replies)
Discussion started by: jayadanabalan
6 Replies

9. Shell Programming and Scripting

To check time stamp in log file and calculate.

Hi Friends, I have the following logfile. i want to make a script for calculate time by time2 - time1 1600266278|random|1|2014-09-19 02:08:56.024|2014-09-19 02:08:59.398|A|B|ROOM|Num0208559970111101788|1|dog|dos 1600266200|random|4|2014-09-19 02:08:06.572|2014-09-19... (2 Replies)
Discussion started by: ooilinlove
2 Replies

10. Shell Programming and Scripting

Collecting all lines between two time stamp from the log

Can you help me to collect the entire logs between two time stamp. The below awk command collecting the logs only if the line has time stamp. awk '$0>=from && $0<=to' from="150318 23:19:04" to="150318 23:55:04" log file 150318 23:19:04 logentries 150318 23:29:04 logentries 150318... (11 Replies)
Discussion started by: zenkarthi
11 Replies
tcpslice(8)						      System Manager's Manual						       tcpslice(8)

NAME
tcpslice - Extracts sections of or merges tcpdump files SYNOPSIS
/usr/sbin/tcpslice [-dRrt] [-w file] [start-time [end-time]] file... OPTIONS
Dumps the start and end times specified by the given range and exits. This option is useful for checking that the given range actually specifies the times you think it does. If the -R, -r, or -t option has been specified, the times are dumped in the corresponding format; otherwise, raw format (-R) is used. Dumps the timestamps of the first and last packets in each input file as raw timestamps in the form sssssssss.uuuuuu. This option can not be specified in conjunction with the -r or -t option. Same as the -R option except the timestamps are dumped in human-readable format, similar to that used by the date(1) command. This option cannot be specified in conjunction with the -R or -t options. Same as the -R option except the timestamps are dumped in tcpslice format, in the ymdhmsu format. See the DESCRIPTION section. This option cannot be specified in conjunction with the -R or -r option. Directs the output to file rather than stdout. DESCRIPTION
The tcpslice program extracts portions of packet-trace files generated using the tcpdump -w command. It can also be used to concatenate files. The tcpslice command copies to stdout all packets from its input file(s) whose timestamps fall within a given range. The starting and end- ing times of the range may be specified on the command line. All ranges are inclusive. The starting time defaults to the time of the first packet in the first input file; this is called the first time. The ending time defaults to ten years after the starting time. Thus, the command tcpslice trace-file copies trace-file to stdout (assuming the file does not include more than ten years' worth of data). There are a number of ways to specify times. The first is using UNIX timestamps of the form sssssssss.uuuuuu (the format specified by the tcpdump -tt command). For example, 654321098.7654 specifies 38 seconds and 765,400 microseconds after 8:51PM PDT, Sept. 25, 1990. The examples in this reference page use Pacific Daylight Time (PDT); however, when displaying times and interpreting times symbolically (as shown in this reference page), tcpslice uses the local time zone, regardless of the time zone in which the tcpdump file was generated. The daylight saving setting used is that which is appropriate for the local time zone at the date in question. For example, times associated with summer months will usually include daylight saving effects, and those with winter months will not. Times may also be specified relative to either the first time (when specifying a starting time) or the starting time (when specifying an ending time) by preceding a numeric value in seconds with a plus sign (+). For example, a starting time of +200 indicates 200 seconds after the first time, and the two arguments +200 +300 indicate from 200 seconds after the first time through 500 seconds after the first time. Times may also be specified in terms of years (y), months (m), days (d), hours (h), minutes (m), seconds (s), and microseconds(u). For example, the UNIX timestamp 654321098.7654 discussed earlier could also be expressed as follows: 1990y9m25d20h51m38s765400u When specifying times using this style, fields that are omitted default as follows: If the omitted field is a unit greater than that of the first specified field, its value defaults to the corresponding value taken from either first time (if the starting time is being specified) or the starting time (if the ending time is being specified). If the omitted field is a unit less than that of the first specified field, then it defaults to zero. For example, suppose the input file has a first time of the UNIX timestamp mentioned previously (38 seconds and 765,400 microseconds after 8:51 PM PDT, September 25, 1990). The following example specifies 9:36 PM PDT on the same date: 21h36m The following example specifies a range from 9:36 PM PDT through 1:54 AM PDT the next day: 21h36m 26d1h54m Relative times can also be specified when using the ymdhmsu format. Omitted fields then default to zero (0) if the unit of the field is greater than that of the first specified field, and to the corresponding value taken from either the first time or the starting time if the omitted field's unit is less than that of the first specified field. Using the first time of the UNIX timestamp mentioned previously, the following example specifies a range from 10:00 PM PDT on that date through 11:10PM PDT: 22h +1h10m The following example specifies a range from 38.7654 seconds after 9:51 PM PDT through 38.7654 seconds after 11:01 PM PDT: +1h +1h10m The first hour of the file could be extracted using the following specification: +0 +1h Note that with the ymdhmsu format there is an ambiguity between using m for month or for minute. The ambiguity is resolved as follows: if an m field is followed by a d field, it specifies months; otherwise it specifies minutes. If more than one input file is specified, tcpslice first copies packets lying in the given range from the first file. It then increases the starting time of the range to lie just beyond the timestamp of the last packet in the first file, repeats the process with the second file, and so on. In this manner, files with interleaved packets are not merged. For a given file, only packets that are newer than any in the preceding files will be considered. This mechanism avoids any possibility of a packet occurring more than once in the output. RESTRICTIONS
An input filename that beings with a digit or a plus sign (+) can be confused with a start and end time. Such filenames can be specified with a leading period and backslash (./); for example, specify the file 04Jul76.trace as The tcpslice program cannot read its input from stdin, since it uses random-access to read through its input files. The tcpslice program does not write to its output to a terminal (as indicated by isatty(3)). This prevents binary data from displaying on a user's terminal. You must either redirect stdout or specify an output file using the -w option. The tcpslice program does not work properly on tcpdump files spanning more than one year with files containing portions of packets whose original length was more than 65,535 bytes or with files containing fewer than three packets. If you use these files, the following error message is displayed: couldn't find final packet in file These problems are due to the interpolation scheme used by tcpslice to significantly increase its processing speed when dealing with large trace files. The tcpslice program can efficiently extract slices from the middle of trace files of any size, and can also work with trun- cated trace files (that is, the final packet in the file is only partially present, typically caused by tcpdump being killed). SEE ALSO
Commands: pfstat(1), pfconfig(8), tcpdump(8) Files: bpf(7), packetfilter(7) tcpslice(8)
All times are GMT -4. The time now is 10:41 AM.
Unix & Linux Forums Content Copyright 1993-2022. All Rights Reserved.
Privacy Policy