UNIX for Dummies Questions & Answers

Boot off a cdrom into single user and move everything back. You won't be using the commands from the disk, all your stuff is still there - of course, I don't know if this works in SCO.

Once upon a time... one of the more experienced/brilliant SA I've known was using an automated update script (cfengine). Instead of moving /etc/passwd out of the way, it moved /etc. Since this was automated, over half the servers were hit before it updated the server that was being used, killing it and stopping the spread of destruction.

After about 6 hours of trying to figure out a way out of this (besides doing rebuilds), one of the Sys Admins that use to work there was called. He came in as a favor to the SAs that were still there. Within five minutes of getting there, he found that everything was still there, just that it was in lost+found. Each server was booted into single user from either a cdrom or the jumpstart server (which was just finished being rebuilt (during the six hours) and the etc directory moved back. What was looking to be a 2 or 3 day outage attempting to rebuild 49 servers was brought to an end in 12 hours (total time from screw up to AOK - 18 hours).

I bet that was a fun couple of days