What is on Your Mind?

This thread reminded me of a couple of screwups made by an operator in my old job. The ops used to run OS backups through a script which asked for the path to the tape (/dev/rmt/whatnot). We (the sysadmins) would give these to the ops if a new server was added or the h/w path changed. One Sunday (when else!) one of the ops gave /dev/rmtwhatnot instead of /dev/rmt/whatnot. ufsdump filled up / and we had to run to the datacenter to take console...

Anyone wants to share one of their stories?

we used to have root equivalent accounts --- regular usernames with UID 0 --- in the old days and we had to get rid of them as audit didn't like them ... anyways, i deleted the accounts on my servers using the userdel method without deleting the home directory and told a colleague of mine that she can do the same ... well, she followed most of my instructions ... she killed the server when she specified to remove the root equivalent user's home directory which in this case was "/" ... had a fun time with server recovery that day ...

A very similar userdel incident was caused by a Junior admin in my last job. We had a few users with their home directories set to /opt/appname. He was cleaning up, and used userdel -r to remove one of these users. I was oncall, and received a call from the helpdesk guys telling me that FooApp had stopped working. Turns out that FooApp didn't like it's entire directory tree removed from underneath it I spent the next few hours restoring from tape

This happened in my previous organization. My Team Lead was a Windows guy. He got transferred to a different Team, where UNIX was used. He once called us, asking how to restore files that were deleted. He deleted some important files (I think, out of ignorance; there is no recycle bin in UNIX).

In a really BIG oil company i worked for several years ago the Unix group (me) got a new team leader - a former systems programmer from the mainframe. One day i hear him curse in his office:

Me:What is the problem?
He: I type in this line now for the fifth time...

He was typing some really long commandline with several long pathnames and of course didn't get it right - he didn't know about commandline editing and the cursor keys were of course not working. I went there, issued "set -o vi" and hit ESC-k, then showed him how to edit the commandline.

Several days later i heard him cursing again:

Me: What's up, need help?
He: The tips you give me are all nonsense! You told me about ESC-k and now see this...

I had a look on his screen, where a shell with the SAP R/3 system user where open, a line full of ESC-k's where there. For no apparent reason SAP had its system user use csh instead of ksh like the rest of the system (AIX 3.2.5). The following dialog took place:

Me: ahh, that is c-shell, use "!!" to repeat the command
He: and why not ESC-k?
Me: because its C-shell
He: and why is ESC-k working here (points to another window)
Me: because it is Korn Shell
He: ok, but why isn't it working here? (point to csh window)
Me: because it is C-shell and not Korn Shell
(...skipped at least 5 repeats ....)

He: so you want to tell me there are *several* shell in Unix??
Me: yes. The common shell in AIX is Korn shell, some prefer BASH, the SAP system user is csh, which some programmers also prefer and there is the ancestor of them all, the Bourne shell and some others, ...

After this he went into his office and started thinking. 2 hours later he came out beaming:

He: I have thought that over. We will now decide which shell we use as a standard and every other shell will have to be deleted from the system.

I told him to discuss this with the IBM technician and resigned from the job within the next hour.

bakunin