SSRF Explained Through Real-World Mistakes

This post started as a note to my future self. Every few months I relearn the same lessons about SSRF, so this time I'm writing them down.

Most of the complexity was accidental. Over the years we'd layered three different approaches to SSRF on top of each other, and each one made sense at the time.

The problem

When we finally measured, the numbers were worse than anyone guessed. Around 29% of the time spent on a typical change went into working around SSRF rather than building anything.

We had good intentions. We even had a document describing how SSRF should work. The trouble was that the document and the code had quietly gone their separate ways.

The fix

Rather than a big-bang rewrite, we migrated one module at a time over about 9 weeks, keeping both paths alive behind a flag.

Here's a trimmed-down version of what we ended up with:

cosign sign --yes ghcr.io/acme/api@sha256:3f1c...
cosign verify \
  --certificate-identity-regexp 'https://github.com/acme/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  ghcr.io/acme/api@sha256:3f1c...

Nothing about this is clever, and that's the point. Anyone on the team can read it in a minute and know exactly what it does.

A few things that helped along the way:

• Keep the old path around until the new one has survived a real incident.
• Agree on what "done" means before the first pull request.
• Pair on the first few changes so the knowledge doesn't live in one head.
• Delete dead code aggressively; it's the cheapest refactor there is.

Caveats

There are cases where this falls apart. If your team is very small or your traffic is very spiky, the extra moving parts may not pay for themselves.

None of this is free. The new approach is a bit more verbose, and newcomers occasionally ask why we don't just use Sigstore for everything. That's a fair question with a boring answer: we tried, and it made SSRF harder to reason about.

Takeaways

That's it for now. Next time I'll dig into secret rotation, which turned out to be closely related.