Before I chuck my couple of cents worth into the bucket here, a quick précis on me and what I’m doing at the moment.
I’m nearing retirement, I’ve worked on a huge range of equipment – for a long list of names, pretty much all gone now. Probably worked on more than 20 flavours of *NIX for companies like Data General, Sun, Olivetti, Norsk Data, Wordplex, Motorola, Intergraph and a number of others.
For the last 15 years I have been a “Data Centre Migration Specialist”, whatever one of those is. At the moment I am sub- contracted to a client by IBM. At this point I should say that I am not permanently employed by IBM, but this is the fourth time that I’ve been contracted out by IBM. The current job is to move the data centre of a major player in the UK utility market into a new headquarters building, a project expected to last at least another 18 months.
The IBM estate is pretty mixed and aged, I have a number of P770’s, P740’s, P570’s and RS6000’s running a number of levels of AIX from 4.3 to 7.1 – with 7.2 about to go on the floor in the form of a number of S824’s – there are a total of four HMC’s. I have also got quite a number of Linux (200) and Sun (350) servers to move, the end client has hardware support from Oracle, IBM and HP-CDS and OS support from IBM and Oracle.
So now my 2˘ worth:-
I can agree with most of what has been said above, I can understand IBM wanting to lock the HMC appliance down as much as possible and I understand the sysadmin desire to have full control of any machine on the network as Bakunin says – if there’s not a competency issue. In truth, my main reason for coming down on the restricted side of this argument is exactly that – competency! I have a number of systems that have been up and running for longer than many of my support contacts have been systems admins, I don’t actually have privileged access to many of the systems – I have elevated access or “root” access on none of the systems. Should I need root access, it has to be requested, approved and I am issued with a one-time password.
I find it to be a total pain, but that is the implemented system. On investigation the reason for the system being implemented was, you guessed it competency! Cited examples, well I could give you any number. But an example that I think sums it up quite well is one that was easy to recover from, but could have been catastrophic had it been a customer facing system with say five or six thousand users. Instead of a development system, with just a couple of hundred developers. Where the “root” user executed a recursive delete command with a space in it, from the root directory and effectively deleted the full contents of the server – mostly source code and development tools.
I have worked in the *NIX world since 1981, over that time I have watched the skill level of the sysadmin degrade, a lot of it revolves around training – my first “Sysadmin I” course was five weeks long and I never actually saw a machine. It was all spent sitting at a Wyse 30 terminal, with a number of other trainees. Now I see sysadmins working for major vendors, with no training whatsoever.
I am in many respects happy that these administrative and management appliances have been made idiot proof as much as possible, but also very wary – just when you find that you have secured the systems against Idiot V1.0, you’ll find that the management will upgrade to Idiot V2.0.
IMHO only training and experience makes for a competent sysadmin, but unfortunately these things come with a high price tag. Inexperienced resource is easy to find and cheap to run, moving the support off shore can exacerbate the problem – through language, not competency although my personal experience has been that you have the same ratio of competent/incompetent people evenly distributed around the world.
I have tried to keep myself current with as much as I can, even attending further training – here I definitely agree with Bakunin. When I’m doing stuff “I want to know what I’m actually doing”, after many years of AIX – and using “smit” on both AIX and Solaris(for information, smit was ported to Solaris by a major financial company in the UK), I knew about pressing F6 to see what was going to be run by the system. The standard of knowledge of the instructor made it obvious that he had almost no experience, as he couldn’t answer some of the simplest questions and answered others incorrectly – at which point I actually asked to see the manager of the training facility to request reimbursement.
So when I see the standard of people moving into the sysadmin world, I can understand why the move to making things safe through idiot proofing. My approach would be to weed out the idiots and provision better training, but unfortunately that costs more.
Gull04