14 DAY TRIAL // Marco Marsala is the owner of a small hosting company, or better yet, was the owner of a small hosting company, which, unfortunately, will go out of business following a costly mistake the developer made.
Two days ago, Mr. Marsala had quite an unpleasant surprise when he found a bug in his code, but only after the damage was already done. As he recounts the events on a tech support site, he managed to accidentally wipe-clean all of his servers, including the backup network, which should have been separated from his normal production infrastructure.
The mistake was due to bad code design in Ansible, a Linux utility that's used to run automated scripts at various intervals on multiple servers simultaneously.
Script error leads to disaster
Mr. Marsala revealed that in one of these scripts he used the mythically dangerous "rm -rf" bash command, which tells a server to delete data by ignoring errors and without asking for any other type of user input.
As the developer explained, the actual syntax was "rm -rf {foo}/{bar}", where foo and bar were two variables that were being dynamically determined and passed around through the script.
Because of an error in the variables' processing, the generic syntax failed to insert their values in the bash command. This command then became "rm -rf /", which means "delete everything without asking me anything" at disk path "/", which is the computer's root (all drives).
Since Mr. Marsala runs a small hosting company with 1,535 customers, this wouldn't have been a problem, since he could always restore the data from backups. It would have taken him a few hours, or even days, but he had the backups to save his business.
Backup data was also deleted by the same script
As it turned out, the Universe was against Mr. Marsala two days ago because the "rm -rf /" command was actually in the backup script itself. Mr. Marsala explained that, just two lines above the problematic command, his servers were instructed to connect the backup network to the main hosting network in order to start transferring data between the two.
This meant that the destructive command ran on both systems, wiping out customer data and their backups.
Despite his plea of help online, nobody was able to provide him with any suggestion on how to recover his data, which means that Mr. Marsala effectively wiped his business off the face of the Earth.
Probably the best piece of advice he got was: "You're going out of business. You don't need technical advice, you need to call your lawyer."
UPDATE: It appears that the person behind the Server Fault topic was nothing more than a sneaky troll.