SysAdmin Rite of Passage
Aug. 29th, 2005 09:28 amGreg, one of my colleagues in an adjacent cube, is having a very hard day today and it's only 9:30 AM.
About 30 minutes ago, he meant to type the following (with root privileges):
crontab -l | grep reboot
What he actually typed was:
crontab -l | reboot
For the non-geek, the first command would have searched through the list of automatically-scheduled commands for the time & date of the automated reboot. The second one listed the commands and initiated the reboot immediately. It's a small typo with big consequences: the unscheduled reboot set off all manner of alarms & automated pages for the system administrator staff, our managers, the business users, the developers, the customer support staff and our senior corporate management. There's a conference call in progress at the moment to examine the causes & consequences of the reboot.
Since Greg is something of a perfectionist, he's dying of embarrassment as I type this, and would like nothing better than have the earth swallow him up.
This significant faux pas is, for our staff at least, something of a rite of passage: every senior admin in our data centers has made this kind of error at least once in his/her career. I've done it myself at least twice in the past decade or so, which isn't a bad average since I support a division of approximately 400 servers. In a twisted kind of way, Greg has finally arrived: he's now truly One Of Us. :-)
It's not much of a comfort at the moment, but we'll have a good laugh about it some day.
About 30 minutes ago, he meant to type the following (with root privileges):
crontab -l | grep reboot
What he actually typed was:
crontab -l | reboot
For the non-geek, the first command would have searched through the list of automatically-scheduled commands for the time & date of the automated reboot. The second one listed the commands and initiated the reboot immediately. It's a small typo with big consequences: the unscheduled reboot set off all manner of alarms & automated pages for the system administrator staff, our managers, the business users, the developers, the customer support staff and our senior corporate management. There's a conference call in progress at the moment to examine the causes & consequences of the reboot.
Since Greg is something of a perfectionist, he's dying of embarrassment as I type this, and would like nothing better than have the earth swallow him up.
This significant faux pas is, for our staff at least, something of a rite of passage: every senior admin in our data centers has made this kind of error at least once in his/her career. I've done it myself at least twice in the past decade or so, which isn't a bad average since I support a division of approximately 400 servers. In a twisted kind of way, Greg has finally arrived: he's now truly One Of Us. :-)
It's not much of a comfort at the moment, but we'll have a good laugh about it some day.