Ugh, I feel for this guy. I once wiped out our data warehouse by overwriting five of the production SAN volumes with backup copies from a test database. I had been, ironically, been working to do the opposite. If you've ever worked with EMC's Symmetrix devices and BCV map files, you know the horrible, horrible hoops you need to jump to map devices under Solaris. I had written the data down one way, then one the next page (after receiving a couple of annoying phone calls) wrote it down another. Luckily, we didn't lose any data, but it was very embarrassing to bring down the main production data warehouse for the day. No one was mad because I fessed up too, which made me feel better. Still, I think we all have our fat-fingering stories...
I've done my share of bone-headed moves too. Most commonly, they've been cut-and-paste errors between the wrong windows, accidentally triggering actions on the wrong server. I've accidentally rebooted a server or two in the past as well. I haven't wiped out whole SAN volumes, but I have been present when it happened and it isn't a good feeling, even vicariously.
Deleting the account of a former sysadmin who instead of using 'su', set all of his logins to UID 0. Didn't catch the UID 0 part and let the script delete his files - kernel panic trying to delete /bin/sh. Ooooooops.
I'd have made the same error... it wouldn't have occurred to me to check as any admin who sets his default uid to 0 deserves to be flogged (IMHO, of course).
no subject
Date: 2007-03-20 05:06 pm (UTC)no subject
Date: 2007-03-20 07:18 pm (UTC)no subject
Date: 2007-03-20 07:19 pm (UTC)My bonehead event.
Date: 2007-03-20 10:34 pm (UTC)Re: My bonehead event.
Date: 2007-03-21 12:41 pm (UTC)