Right On Time, Somewhere


Login or Register to Reply

 
Thread Tools Search this Thread
# 1  
Old 01-12-2012
Right On Time, Somewhere

Things like this are teaching me a greater appreciation for network time...

The huge gap between the speed of a satellite connection and its tiny bandwidth allowance means either being completely draconian or having to chide people to not abuse it all the time. We're somewhere inbetween, if a customer downloads too much they'll be slowed down temporarily, but not shut off; if the situation continues, we may phonecall and investigate.

We had a situation where a small community had massively overused their satellite connection pretty much collectively, going over not only our limits but our provider's limits, causing the entire satellite connection to be throttled. We needed to shut the connection down for a few hours before the satellite modem would let go.

We planned it, set a time, and warned our customers. How it worked was very simple -- two entries in root's crontab. At noon that day, the first one would run 'ifconfig eth1 down', taking the community offline but leaving me in communication with the server. At 5pm that day, it would run '/sbin/reboot'.

The server clock had drifted far more than I'd anticipated in the months since its last boot and clock-set, and the shutdown happened one hour early.
# 3  
Old 01-13-2012
Testing it in dev first. If I mess up a production system, it's hours of driving and sometimes a bit of sledding to fix them in person, especially at this time of year.
# 4  
Old 01-16-2012
Smilie Thanks for sharing your story. It's very true that most of the times we do not bother to check the time of the clock before scheduling stuffs.

We maintain IT infrastructure for a big pharma company. For any SLA (service level agreement) breach, my employer has to pay a real big amount of money to the client. Now that's been told, once my colleague had to schedule a maintenance on an AIX server. We have a procedure to do that. There's a lot of approvals from service delivery managers of both the client and our company required. After getting those, this guy went on scheduling the reboot of the machine in maintenance mode in cron a day before. The next day, I got a call from IT Incident management people saying a server is down before it's scheduled maintenance window. It happened around 20 minutes before the scheduled time. We had to raise a severity for this. Upon checking the root cause of this later, we found somehow the server was failing to sync with the NTP server and the clock was going 20 minutes faster than the actual time.

And yes, because of all these, we breached the SLA! Smilie
This User Gave Thanks to admin_xor For This Post:
nmatt00 (01-30-2012)
Login or Register to Reply

|
Thread Tools Search this Thread
Search this Thread:
Advanced Search

More UNIX and Linux Forum Topics You Might Find Helpful
Calculate Time diff in milli milliseconds(Time format : HH:MM:SS,NNN) Raza Ali Shell Programming and Scripting 6 03-06-2018 09:55 AM
Find gaps in time data and replace missing time value and column 2 value by interpolation in awk malandisa Programming 4 01-16-2015 11:11 AM
Convert UTC time into current UNIX sever time zone mohanalakshmi Shell Programming and Scripting 5 12-11-2013 08:19 PM
Displaying current date time of EDT in IST time Showdown Shell Programming and Scripting 6 03-16-2013 11:17 AM
Adding time to date time in UNIX shell scipting codehelp04 Shell Programming and Scripting 3 12-03-2012 05:31 AM
modifying date and time and time zone on solaris 5.10 with (redundant server) veritas barry1946 Solaris 1 09-20-2012 01:28 AM
Converting string date time to unix time in AWK bkkid UNIX for Dummies Questions & Answers 2 08-19-2012 05:06 PM
How to get time duration between two human readable time stamp in Unix? ford99 Shell Programming and Scripting 3 07-19-2011 10:52 AM
Shell script to convert epoch time to real time aismann Shell Programming and Scripting 4 04-25-2011 11:09 PM
How to calculate time difference between start and end time of a process! smarty86 Shell Programming and Scripting 16 09-03-2010 04:15 PM
Convert Epoch Time to Standard Date and Time & Vice Versa DrivesMeCrazy Shell Programming and Scripting 5 02-07-2009 01:40 AM
Start time/end time and status of crontab job thambi Shell Programming and Scripting 3 05-16-2007 11:24 AM
How To Provide Time Sync Using Nts-150 Time Server On Unix Network? pesty UNIX for Advanced & Expert Users 2 03-22-2007 03:20 AM