Menu
Comingdeer Blog
  • About Me
  • Contact Me
Comingdeer Blog

Zero Trust in AWS Part 2: The $250K Wake-Up Call

April 19, 2025January 10, 2026

This is Part 2 of my Zero Trust series. If you haven’t read Part 1: How I Learned to Stop Worrying and Love Paranoid Security, start there for the foundation.

In Part 1, I talked about Zero Trust like it was a nice-to-have security philosophy. Layer cakes and paranoid parents. But let me tell you about a customer who learned why Zero Trust isn’t optional—the hard way.

The Breach: A Friday Night Horror Story

It started on a Friday evening, because of course it did. A root user password was compromised—phishing, credential stuffing, a reused password from a breached database. The “how” matters less than what happened next.

Within hours, the attacker had spun up over $250,000 worth of EC2 instances across multiple regions. Crypto mining at full blast on someone else’s dime. But that wasn’t the worst part.

The attacker systematically deleted production resources. Databases. Application servers. Storage volumes. They didn’t just steal—they burned the house down on the way out. By Saturday morning, the customer’s entire production environment was gone.

The AWS bill was catastrophic. But the real damage went deeper.

The Fallout Nobody Talks About

The $250K in fraudulent compute charges was just the beginning. Emergency incident response, forensic investigation, customer notification, legal consultation, engineering hours to rebuild—the total cost approached half a million dollars.

Then came the reputational damage. Two major enterprise contracts were paused pending security review. One never resumed. The sales team spent months rebuilding trust that took years to establish.

Remember what I said in Part 1? It takes years to build security trust with your customers, but it only takes 5 minutes of you being hacked to break all that trust.

This customer lived that reality.

The Recovery: Zero Trust Gets Real

Here’s the silver lining: they had backups. A separate Disaster Recovery account—completely isolated from production—contained everything needed to rebuild. That decision saved the company.

When we deployed Platformr and rebuilt their environment, Zero Trust got real:

Service Control Policies as the outer guardrail. SCPs at the Organization level now prevent anyone—including root—from disabling CloudTrail, deleting backups, or launching resources in unapproved regions. Even if credentials get compromised, the blast radius is contained.

Multi-account IAM with permission boundaries. Each workload lives in its own account with explicit cross-account roles. Permission boundaries cap what any IAM policy can grant, so even an admin can’t escalate beyond defined limits. Root credentials? Locked in a vault with hardware MFA and break-glass procedures.

IAM Access Analyzer running continuously. Every policy gets analyzed for unintended external access. That S3 bucket someone made public for “testing”? Flagged within minutes. Cross-account trust relationships validated against an approved list—anything unexpected surfaces immediately.

CloudTrail with teeth. Logs ship to a separate security account where even Organization admins can’t delete them. Anomaly detection triggers on root logins, console access from new locations, and resource creation spikes. The crypto mining that ran for hours? It would’ve triggered alerts within minutes.

The Result

Six months later, this customer passed their SOC 2 audit with zero findings. Their enterprise customers now cite their security posture as a competitive advantage.

The difference between “before” and “after” isn’t technology. It’s philosophy. They stopped assuming their perimeter would hold and started assuming breach. They stopped trusting and started verifying.

Zero Trust isn’t paranoia. It’s preparation. And preparation is a lot cheaper than $250K in crypto mining charges and a Saturday spent explaining to customers why their data is gone.

Tell me what you think:



Comments

Recent Posts

  • Teaching My Daughters About Risk (While Letting Them Take Their Own)
  • Stop Asking Your Senior Architect to Write Unit Tests: A Guide to AI Team Management
  • Managing AWS Organizations Top-Down to Eliminate Hidden Costs
  • That Landing Zone You Built 3 years ago? It’s Already Old.
  • Zero Trust in AWS Part 2: The $250K Wake-Up Call

Tags

ai Audible AWS build vs buy Cancer Church Software Cloudwatch Synthetics Consulting CTO Employees Evidence Five Talent Frugality Goals Health HR Identity Crisis Insanity Jesus Kids Kissing kpis Lambda Landing Zones Leadership Managers Marriage Money MVP My Purpose obfuscation OKRs Parenting Private Time Risk Security Self-Employment Selflessness serverless software development staging Steve Timeshares Wings of Fire Wordpress

Archives

  • January 2026
  • December 2025
  • October 2025
  • May 2025
  • April 2025
  • November 2024
  • September 2024
  • August 2024
  • February 2024
  • December 2023
  • November 2023
  • October 2023
  • March 2023
  • February 2023
  • December 2022
  • September 2022
  • June 2022
  • May 2022
  • April 2022
  • January 2022
  • December 2021
  • November 2021
©2026 Comingdeer Blog