Unix/Linux Go Back    


War Stories Tell your work related tech stories and share experiences here. Share your successes and failures and other "war stories" in this forum.

Prize of being an Admin

War Stories


Closed    
 
Thread Tools Search this Thread Display Modes
    #1  
Old Unix and Linux 02-07-2012   -   Original Discussion by admin_xor
admin_xor's Unix or Linux Image
admin_xor admin_xor is offline
Registered User
 
Join Date: Jun 2011
Last Activity: 3 May 2015, 12:12 PM EDT
Posts: 452
Thanks: 13
Thanked 80 Times in 76 Posts
Prize of being an Admin

Was wondering if anyone has come across any situation where you do your best to help users and in return you get a nice escalation from top level management!

Here's my story:

One fine morning, I was sitting idle, doing next to nothing, I got an alert from helpdesk people about a problem with an application running on a high-priority AIX LPAR (sorry for not being specific as I am not allowed to disclose the details). This application is actually used heavily by the bio-tech research fellows. The application was crashing frequently. Upon checking in-depth, I found there's nothing wrong from the OS stand point. I also found the problem was caused by one rouge wrapper script. But as a server admin, I had nothing to do with that officially. In the mean time, I had to speak to the manager of the research team and had to try my best to make her understand where the actual problem was. I assured her that I would contact the responsible application admin team.

Followed by that, I tried contacting that application admin team (which is a different IT services vendor than my company). I did not get any solid reply from them nor any definite timeline when they would be able to look into the problem.

I ended my shift. Still no reply from the application team. I came home. Gave it a lot of thought whether I should modify the script by myself. I accessed the server from home, spent a sleepless night understanding how everything of that application was wired.

Next morning, I went to office, saw my E-mail inbox hoping that the app team might have responded. But no! So I went ahead, made the changes that I thought should have been done. restarted the application service. Monitored the application for a couple of hours and called up the manager of the research team. She was furious. I politely told what I had done so far and asked her to check if things were fine. And yes, things were fine. But that lady was quite upset about our service.

After 6 hours, I saw an E-mail from our top level Service Delivery Manager demanding an answer as to why it took so long for me to fix the issue. Upon checking I found the research team manager had sent a "beautiful" E-mail to the top level manager saying how worthless people in IT were.

After two days, my supervisor showed up after his vacation. After knowing my deed, all he had to ask was "Why the hell did you even think of taking such bold step? What if something wrong had happened?". Being one of the junior members in the team, I kept quiet. All I tried to do was to help the research folks out and let them continue their work as soon as possible. But I got the prize of doing that. Linux
Sponsored Links
    #2  
Old Unix and Linux 02-07-2012   -   Original Discussion by admin_xor
zxmaus's Unix or Linux Image
zxmaus zxmaus is offline Forum Advisor  
Event Manager
 
Join Date: May 2008
Last Activity: 7 September 2014, 5:24 AM EDT
Location: in a putty session :)
Posts: 811
Thanks: 19
Thanked 71 Times in 62 Posts
so you learned your lesson: if you fix things that are broken but not owned by you - don't tell anyone Linux
The Following 2 Users Say Thank You to zxmaus For This Useful Post:
Sun Fire (06-12-2012), tayyabq8 (10-12-2015)
Sponsored Links
    #3  
Old Unix and Linux 02-08-2012   -   Original Discussion by admin_xor
Neo's Unix or Linux Image
Neo Neo is offline Forum Staff  
Administrator
 
Join Date: Sep 2000
Last Activity: 14 December 2017, 1:27 AM EST
Location: Asia pacific region
Posts: 14,091
Thanks: 934
Thanked 1,270 Times in 608 Posts
I agree with zxmaus and the old saying:

"No good deed goes unpunished."

When in a political (territorial) situation as described; just fix the problem and don't tell anyone you fixed it, since you are the main system admin with root privs.

"No good deed goes unpunished."

.. as they say....
The Following User Says Thank You to Neo For This Useful Post:
tayyabq8 (10-12-2015)
    #4  
Old Unix and Linux 02-08-2012   -   Original Discussion by admin_xor
admin_xor's Unix or Linux Image
admin_xor admin_xor is offline
Registered User
 
Join Date: Jun 2011
Last Activity: 3 May 2015, 12:12 PM EDT
Posts: 452
Thanks: 13
Thanked 80 Times in 76 Posts
Well said zxmaus and Neo! Lesson learned!!

I used to think why all of the senior admins were so lifeless. And here I am, always ready to dive into any problem with extra zeal!! But now I know, I need to be controlled and there's a reason why seniors are indifferent on things. Linux
Sponsored Links
    #5  
Old Unix and Linux 02-08-2012   -   Original Discussion by admin_xor
bartus11's Unix or Linux Image
bartus11 bartus11 is offline
Registered User
 
Join Date: Apr 2009
Last Activity: 3 August 2016, 11:03 AM EDT
Posts: 3,733
Thanks: 7
Thanked 1,153 Times in 1,123 Posts
I think what you should have done is "shift (or share) responsibility" ie - ask research team manager if you can try fixing the issue by yourself if the application responsible is not responding for few hours. Then you would avoid getting trashed by her, as she couldn't say that you wasn't working on the issue. It would also solve the problem of responsibility (at least to some extent) in case things went ugly, that your manager told you about.
Sponsored Links
    #6  
Old Unix and Linux 02-08-2012   -   Original Discussion by admin_xor
admin_xor's Unix or Linux Image
admin_xor admin_xor is offline
Registered User
 
Join Date: Jun 2011
Last Activity: 3 May 2015, 12:12 PM EDT
Posts: 452
Thanks: 13
Thanked 80 Times in 76 Posts
Excellent point, bartus11.
Being a naive and over-enthusiastic one, I was only thinking about the problem on hand, not about the consequences. Looks like I have to work on so many things!! Linux
Sponsored Links
    #7  
Old Unix and Linux 02-08-2012   -   Original Discussion by admin_xor
methyl's Unix or Linux Image
methyl methyl is offline
Registered User
 
Join Date: Mar 2008
Last Activity: 8 June 2016, 9:58 PM EDT
Posts: 6,402
Thanks: 288
Thanked 676 Times in 645 Posts
Difficult situation admin_xor . You dealt with the situation well on the back of an emergency ... but did you have the authority to make the change?
In a formal environment you needed authority from the duty Change Control Manager to go ahead with an Emergency Change.
This could have backfired big time.
Take care.

(I've had a Director out of bed to authorise a minor change when the duty Change Control Manager did not respond).

Last edited by methyl; 02-08-2012 at 07:14 PM.. Reason: stupid typo!
The Following User Says Thank You to methyl For This Useful Post:
marks1a (07-18-2013)
Sponsored Links
Closed

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Linux More UNIX and Linux Forum Topics You Might Find Helpful
Thread Thread Starter Forum Replies Last Post
Windows Admin switching to *nix Admin bobwilson What is on Your Mind? 3 03-10-2011 07:54 AM
Any EVA admin's out there? mr_manny HP-UX 3 06-08-2006 10:47 AM



All times are GMT -4. The time now is 08:52 AM.