Was wondering if anyone has come across any situation where you do your best to help users and in return you get a nice escalation from top level management!
Here's my story:
One fine morning, I was sitting idle, doing next to nothing, I got an alert from helpdesk people about a problem with an application running on a high-priority AIX LPAR (sorry for not being specific as I am not allowed to disclose the details). This application is actually used heavily by the bio-tech research fellows. The application was crashing frequently. Upon checking in-depth, I found there's nothing wrong from the OS stand point. I also found the problem was caused by one rouge wrapper script. But as a server admin, I had nothing to do with that officially. In the mean time, I had to speak to the manager of the research team and had to try my best to make her understand where the actual problem was. I assured her that I would contact the responsible application admin team.
Followed by that, I tried contacting that application admin team (which is a different IT services vendor than my company). I did not get any solid reply from them nor any definite timeline when they would be able to look into the problem.
I ended my shift. Still no reply from the application team. I came home. Gave it a lot of thought whether I should modify the script by myself. I accessed the server from home, spent a sleepless night understanding how everything of that application was wired.
Next morning, I went to office, saw my E-mail inbox hoping that the app team might have responded. But no! So I went ahead, made the changes that I thought should have been done. restarted the application service. Monitored the application for a couple of hours and called up the manager of the research team. She was furious. I politely told what I had done so far and asked her to check if things were fine. And yes, things were fine. But that lady was quite upset about our service.
After 6 hours, I saw an E-mail from our top level Service Delivery Manager demanding an answer as to why it took so long for me to fix the issue. Upon checking I found the research team manager had sent a "beautiful" E-mail to the top level manager saying how worthless people in IT were.
After two days, my supervisor showed up after his vacation. After knowing my deed, all he had to ask was "Why the hell did you even think of taking such bold step? What if something wrong had happened?". Being one of the junior members in the team, I kept quiet. All I tried to do was to help the research folks out and let them continue their work as soon as possible. But I got the prize of doing that.
I used to think why all of the senior admins were so lifeless. And here I am, always ready to dive into any problem with extra zeal!! But now I know, I need to be controlled and there's a reason why seniors are indifferent on things.
I think what you should have done is "shift (or share) responsibility" ie - ask research team manager if you can try fixing the issue by yourself if the application responsible is not responding for few hours. Then you would avoid getting trashed by her, as she couldn't say that you wasn't working on the issue. It would also solve the problem of responsibility (at least to some extent) in case things went ugly, that your manager told you about.
Difficult situation admin_xor . You dealt with the situation well on the back of an emergency ... but did you have the authority to make the change?
In a formal environment you needed authority from the duty Change Control Manager to go ahead with an Emergency Change.
This could have backfired big time.
(I've had a Director out of bed to authorise a minor change when the duty Change Control Manager did not respond).
Last edited by methyl; 02-08-2012 at 07:14 PM..
Reason: stupid typo!
The Following User Says Thank You to methyl For This Useful Post: