Understanding Unix Timekeeping | Unix Linux Forums | Tips and Tutorials

  Go Back    


Tips and Tutorials Helpful articles from our Users.

Understanding Unix Timekeeping

Tips and Tutorials


Tags
clock, mtime, timekeeping

Closed Thread    
 
Thread Tools Search this Thread Display Modes
    #1  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
Understanding Unix Timekeeping

Internal Time
Internally, a Unix system maintains time as the number of seconds since the Unix Epoch. The Unix Epoch was at midnight Jan 1, 1970 UTC. In the Eastern Timezone of the US that was Dec 31, 1969 at 19:00. As I start this article, the time is 1171733878. I am using NTP (Network Time Protocol) to keep my computer's clock in sync. So at the exact instant that my computer thought it was 1171733878, probably so did yours. Every computer should be in agreement as to what time it is. It does not matter in which timezone your computer resides or in which timezone you reside. If I had written to a file during that second, Unix would have recorded the modification time as 1171733878.

Unix time was traditionally a signed 32 bit integer. That means that maximum time is 2147483647 which, in the Eastern TimeZone of the US would be Jan 18, 2038 at 22:14:07. The plan is for this to expand to a 64 bit integer before then. Unix seconds count forward one-by-one. After 1171733878, the next second that my system will know about is 1171733879. Once it is syncronized, NTP works by shortening or lengthening a second. I can obtain the time by calling the system call time(). If I want something better than seconds, the next most accurate interface is the system call times(). times() main purpose is to obtain a structure of data pertaining to system usage. But the return code is also useful... it is the time in clock ticks. Clock ticks start at some arbitrary point in the past (often boot time) and they advance whenever the kernel clock routine runs. How many clock ticks per second can vary. You can call sysconf(_SC_CLK_TCK) to find out your value. 100 is common value these days. Older systems used 60. 100 and 60 are the only two values I have seen. Clock ticks are also returned as a signed integer. So after about a year, the number will roll back to zero. Other than rollover, clock ticks also advance one-by-one. (Clock ticks are often called "jiffies" by Linux folks.) There is also another system call, gettimeofday() which returns the time since the Unix Epoch in seconds and microseconds. gettimeofday() is required to exist and to use microsecond precision. However the accuracy is explicitly unspecified. The microseconds do not advance one-by-one in any system that I have seen. I assume that Unix systems are fairly good with hundreths of a second and maybe even milliseconds is some cases. But I would not assume any accuracy beyond that.

*** Update ***
Well now the Unix time is 1304431209 and what a difference a few years makes! First, GPS clocks are now available for few thousand US dollars. These clocks have a crystal controlled oscillator as their internal clock. These oscillators vary in rate based on temperature. By stabilizing their temperatures we can stabilize their rates. Cooling them down is hard, so these new GPS clocks ovenize the oscillators to keep them warm. A GPS clock in the US can typically see 6 to 8 GPS satellites. Each satellite has several atomic clocks on board that are monitored and adjusted by the US Naval Observatory. A decent GPS clock will know the time to within 100 nanoseconds, and at this writing, that is a conservative estimate. Today all computers are 64 bits and that really helps NTP keep clocks accurate. And the NTP software itself has advanced over the past few years. At each tick (or jiffy) the amount to increment the system clock is calculated in 64 bits and is accurate to within a few nanoseconds. Today's computers run fast. Most of our servers run at 3 GHZ! Most architectures have a register called a Processor Cycle Counter or PCC that increments with each cycle. NTP reads the PCC and scales it back to nanoseconds and uses the result measure the time between clock interrupts (ticks or jiffies). The PCC is also used to interpolate the current time between ticks. So, yes folks, the clocks are more accurate now than they were a few year ago. How accurate?

NTP can provide the estimated clock error. I am running a script to interrogate this value on all of our workstations and servers as I type. My workstation has estimated error of 36 microseconds (us). Counting only the systems running RedHat on Intel chips the estimated error is ranging from 25 us to 2952 us. It's rare to find a system with a full millisecond of estimated error and 3 milliseconds seems to our worst result. Our legacy Solaris boxes are not so accurate. They range from 31600 us to 57584 us and this the kind of accuracy I had in mind when I wrote this article originally.

*** end of update ***


Human Readable Timestamps
Now that we know how Unix keeps time, let's move on to a different subject: How to display time to a user. Few users will be happy with a timestamp being displayed as 1171733878. Since I am in the Eastern Timezone of the US, I would like to see something like 12:37:58 on Feb 17, 2007. But a person in Los Angeles California would want something like 9:37:58 on Feb 17, 2007. I wrote a little perl script to display the formatted time when given a number like 1171733878. Here is the output as it should appear in both the Eastern and Pacific timezones for a few carefully selected timestamps:
Eastern:
1173596399 Sun 2007-03-11 01:59:59 Standard Time
1173596400 Sun 2007-03-11 03:00:00 Daylight Saving Time
1173607199 Sun 2007-03-11 05:59:59 Daylight Saving Time
1173607200 Sun 2007-03-11 06:00:00 Daylight Saving Time
Pacific:
1173596399 Sat 2007-03-10 22:59:59 Standard Time
1173596400 Sat 2007-03-10 23:00:00 Standard Time
1173607199 Sun 2007-03-11 01:59:59 Standard Time
1173607200 Sun 2007-03-11 03:00:00 Daylight Saving Time
Notice that the internal Unix time does not change with the onset of Daylight Saving Time. What happens is that a new rule for how to display the time takes effect. In order to convert internal time to a timestamp, Unix needs to know which timezone to use. And it needs to know if Daylight Saving is in effect. There are various subroutines that can be called to format a timestamp. All of these eventually invoke tzset() to determine the timezone and the state of Daylight Saving. And tzset always checks for the environment variable TZ to figure out what the user wants. If you don't a TZ environment variable, tzset() will go with some sort of default that can vary from system to system. The TZ environment variable is often set to something like EST5EDT4 which is what I use. This says that I am in a timezone called EST which is offset from Universal Time by 5 hours and during Daylight Saving days my timezone is called EDT which is offset from Universal Time by 4 hours. To know when Daylight Saving is in effect, tzset will look in some data file. Which file (and what format the file is in) varies from system to system. The patches that are available for this year's change in Daylight Saving should really be little more than some revisions to these files.

*** update ***
Also let me mention that you can use perl to convert the time from Unix seconds mode to human readable:
Code:
$ perl -e '@d=localtime (shift); printf "%4d-%02d-%02d %02d:%02d:%02d\n", $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'  0
1969-12-31 19:00:00
$ perl -e '@d=localtime (shift); printf "%4d-%02d-%02d %02d:%02d:%02d\n", $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'  1171733878
2007-02-17 12:37:58
$ perl -e '@d=localtime (shift); printf "%4d-%02d-%02d %02d:%02d:%02d\n", $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0]'  1304431209
2011-05-03 10:00:09
$

And notice that Neo has added the Unix time to the top of every page.

*** end update ***

Universal Time
In addition to knowing about your timezone, a Unix System must be able to use Universal Time (which is almost the same as the old Greenwich Mean Time). This is what "date -u" does. The definition for Universal Time is well known and does not involve daylight saving changes. The Universal Time definition may be build into the date routines rather than using the data file for tzset(). This may be true even if an entry is present for UTC. Mail programs often use UTC timestamps.
The Following User Says Thank You to Perderabo For This Useful Post:
imperiusdamian (12-23-2012)
Sponsored Links
    #2  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
Daylight Saving Ruled Have Changed And I have No Patch....What To Do?

Let's say my system is still set up for last year's rule. My system will soon show the wrong time. What should I do? One thing that is not a very good idea would be to crank the clock forward an hour manually. If I do that my system will have the wrong internal time. Some network based services such as NFS use the internal time. My system would also no longer have the correct Universal Time. NTP (Network Time Protocol) will malfunction and my system's clock will drift. Even for my own timezone, I would have timestamps like 17:00 EST when I should have 17:00 EDT. A few weeks later, the old rule will take effect and I will now need to adjust the clock manually again.

I hope that I have convinced you to not reset the time in this manner. One better option would be to research my system's data file that defines the daylight saving rules for my timezone and edit it myself. This should be a straight forward change. But each system may have its own format.

There does seem to be a near universal solution: use a more complex format for the TZ environment variable. You can actually put the Daylight Saving rule in TZ and this behavior is mandated by Posix. Below is a table of TZ settings that I believe to be correct for the year 2007.
Code:
# Posix Format
TZ="EST5EDT4,M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Eastern
TZ="CST6CDT5,M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Central
TZ="MST7MDT6,M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Mountain
TZ="PST8PDT7,M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Pacific
TZ="AST9ADT8,M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Alaska
TZ="HST10"                                       # US Hawaii

# Old System V Release 3.1 and Xenix Format
TZ="EST5EDT4;M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Eastern
TZ="CST6CDT5;M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Central
TZ="MST7MDT6;M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Mountain
TZ="PST8PDT7;M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Pacific
TZ="AST9ADT8;M3.2.0/02:00:00,M11.1.0/02:00:00"   # US Alaska
TZ="HST10"                                       # US Hawaii

Sponsored Links
    #3  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
Special Considerations For cron

cron Will Respond To Changes in Time
If you adjust the system time while cron is running, cron will notice and attempt to compensate. If time moved backwards, cron will stop running jobs and wait for the clock to advance back to where cron run the last job. If time moved forwards, cron will try to catch up. It will click though each minute continuously until it has run all of the jobs scheduled during the skipped time duration. If you are simply tweaking the system clock because it was off a few seconds, this is perfect. But if the time is a year off, it will probably not be a good thing. For large adjustments to the clock you should kill and restart cron.

cron Will Respond To Daylight Saving Changes
Suppose I have a job scheduled to run at 2:15 Sunday morning. When we "spring ahead", there won't be a 2:15. cron will note that we "sprung ahead" by 60 minutes, so it will add 60 minutes to my job time getting 3:15. It will run the job at 3:15...unless the job was already scheduled to run at both 2:15 and 3:15. In that case, my job will run once at 3:15.

Later in the year, we "fall back". On that Sunday, it will be 2:15 twice. My job will only run at the first 2:15. If my job is scheduled to run at 15 minutes after every hour via the explicit use of an asterisk in the hour field my job will run at both 2:15's. This is not true if I use a range or a list of hours to run. Only jobs with an asterisk for the hour field run at both 2:15's.

cron Has Its Own Timezone
If I schedule my job for 2:15 in the morning, cron will run it when it thinks that the time is 2:15. cron is closely related to the "at" command. If I schedule at job via "at" to run at 2:15, my TZ variable is inspected and the job will run at my 2:15. But cron does not work that way.
    #4  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
Leap Seconds

The definition of a second is carefully fixed. Each second is exactly as long as any other second. When scientists were tying down the precise value of a second, they matched it as exactly as possible to the year 1900. There are 24*60*60=86400 seconds in a day. But the Earth has slowed down a bit mostly due to tidal forces. Every year it needs about .7 seconds extra. Every now and then we have a day with 86401 seconds. Unix tries to pretend that this is not the case. If you are not running NTP, then you must periodically manually tweak your clock and you will compensate for a leap second during your next manual adjustment.

If you are running NTP, Unix tries to make the last second of the day with a leap second last for 2 seconds. With many implementations, the second will inappropriately advance in the middle of this special long second and will be adjusted backwards a few milliseconds later.

So ignoring the potential messy second-long transition, we treat each day as if it was 86400 seconds long.

By the way, it took about a century for the Earth to slow down enough to need .7 seconds extra each year. It will take another century for to slow down enough to need about 1.4 seconds extra year. Do not think that each year is .7 seconds longer than its immediate predecessor.
Sponsored Links
    #5  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
My Perl script

Here is my Perl script that I used in this article...
Code:
#! /usr/local/bin/perl


sub printtime($)
{
        @d=localtime $_[0];
        printf "%12d  %3s %4d-%02d-%02d %02d:%02d:%02d %s\n", $_[0],
                (Sun,Mon,Tue,Wed,Thu,Fri,Sat)[$d[6]],
                $d[5]+1900,$d[4]+1,$d[3],$d[2],$d[1],$d[0],
                ("Standard Time","Daylight Saving Time")[$d[8]];
}

while (do {print "Enter val - "; chomp($val = <>)}) {
        print "val = ", $val, "\n";
        printtime $val;
        printtime $val+1;

}
print "\n";

This is how I verify that the timezone data is correct. I just calculate by hand the second when I expect the Daylight Saving to toggle and I use this script to see if it does...
Sponsored Links
    #6  
Old 02-20-2007
Perderabo's Avatar
Perderabo Perderabo is offline Forum Staff  
Unix Daemon (Administrator Emeritus)
 
Join Date: Aug 2001
Last Activity: 31 July 2014, 10:21 PM EDT
Location: Ashburn, Virginia
Posts: 9,899
Thanks: 59
Thanked 416 Times in 250 Posts
Miscellaneous Odds and Ends

Your OS Imposes Limits On The Range of Dates That It Can Format
You need to check the documentation for your system to which dates it can handle.

For example, HP-UX 11i Version 2 says:
The minimum date supported by mktime() in both 32-bit and 64-bit HP-UX is Friday December 13 20:45:52 UTC 1901. The maximum dates supported by mktime() are Tuesday January 19 03:14:07 UTC 2038 and Friday December 31 23:59:59 UTC 9999 in 32-bit HP-UX and 64-bit HP-UX, respectively. If the calendar time cannot be represented, the function returns the value (time_t)—1 and sets errno to ERANGE. Note the value (time_t)—1 also corresponds to the time 23:59:59 on Dec 31, 1969 (plus or minus time zone and Daylight Saving Time adjustments). Thus it is necessary to check both the return value and errno to reliably detect an error condition.

Solaris 10 says:
The zoneinfo timezone data files do not transition past Tue Jan 19 03:14:07 2038 UTC. Therefore for 64-bit applications using zoneinfo timezones, calculations beyond this date might not use the correct offset from standard time, and could return incorrect values. This affects the 64-bit version of localtime(), localtime_r(), ctime(), and ctime_r().

Nanosecond Resolution
I notice that Posix is specifying a variety of nanaosecond resolution routines. I see that Solaris 10 has implemented them. I have no experience with them and I have ignored them for this thread. The clock_settime(3RT) man page says:
Quote:
A clock_id of CLOCK_HIGHRES represents the nonadjustable, high-resolution clock for the system. For this clock, the value returned by clock_gettime(3RT) represents the amount of time (in seconds and nanoseconds) since some arbitrary time in the past; it is not correlated in any way to the time of day, and thus is not subject to resetting or drifting by way of adjtime(2), ntp_adjtime(2), settimeofday(3C), or clock_settime(). The time source for this clock is the same as that for gethrtime(3C).
This implies that the nanosecond resolution routines are not intended for timekeeping. On the other hand, I see ls now has a -E option to display file times in nanoseconds.

If you display the time in nanoseconds, be sure to keep your eyes very close to the screen. Light cannot travel even a full foot in a nanosecond.

Further Reading
Be sure to read mtime, ctime, and atime for information about the timestamps of files.
Sponsored Links
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Timekeeping in Linux question ... newlinuxuser1 UNIX for Dummies Questions & Answers 31 04-30-2011 05:13 PM
Timekeeping problem maniac173 UNIX for Dummies Questions & Answers 5 02-23-2011 07:47 PM
Really understanding Linux/Unix-Derivatives shitson UNIX for Dummies Questions & Answers 4 09-09-2010 12:11 PM
Understanding UNIX/Linux hbradshaw UNIX for Dummies Questions & Answers 0 04-02-2009 03:41 PM
Need Help Understanding a Unix Command chris86 UNIX for Dummies Questions & Answers 6 10-10-2006 03:35 PM



All times are GMT -4. The time now is 05:50 AM.