""We use AWS EC2 to run ephemeral build agents for continuous integration," he told The Register. "It may spin up a dozen instances per day and shut them down when they're no longer used." Chase and his fellow developers have used this technique for about ten years, and their AWS bills have been predictable - usually between $1,000 and $2,000 each month. They therefore fell out of the habit of regularly scrutinizing their bill."
""Then I made a small mistake that cost us a ton of money," Chase admitted. While updating one of his crew's Amazon Machine Images - a template for a virtual appliance that runs in AWS - he accidentally unchecked an option to delete cloud storage volumes when the instance terminated. The update didn't alter how the AMIs operated. But leaving that box unticked meant that when instances shut down, the storage volumes they used persisted instead of disappearing."
"Two months after his mistaken click, he had therefore unwittingly created hundreds of volumes, each consuming 100 GB and earning AWS a pretty penny. "I didn't suspect anything was wrong until I got an email from the organization that sponsors our AWS usage," Chase told Who, Me? "They wanted to know what changed because our cloud charges for the last two months were $40,000.""
The team uses AWS EC2 ephemeral build agents for continuous integration and typically runs about a dozen instances per day with monthly bills of $1,000–$2,000. A developer accidentally unchecked the option to delete storage volumes on instance termination when updating an AMI. As instances terminated, the attached volumes persisted, creating hundreds of 100 GB volumes over two months. Cloud charges rose sharply to about $40,000. Amazon forgave 40% of the charges, but the sponsoring organization queried the change. The error illustrates the financial risk of unattended cloud resource configurations and the importance of regular billing audits.
Read at Theregister
Unable to calculate read time
Collection
[
|
...
]