There's a number making the rounds right now that should worry anyone shipping software: AI-generated code introduces 1.7x to 2.7x more defects than code written by hand. A SmartBear survey from May found that 70% of software leaders say application quality has already degraded as AI accelerates development.
The math is intuitive once you stop to think about it. AI coding assistants are very good at producing plausible code fast. They're not as good at producing correct code. And they're terrible at knowing the difference — which is, historically, what testing is for.
Before Copilot and friends, writing code was the slow part. Testing lagged behind because there was never enough time to test everything you wrote. Now writing is nearly instant, and testing is the only thing standing between a feature and a production incident.
Sixty percent of the teams in that SmartBear survey said quality suffered specifically because code creation outpaced testing capacity. The code got written, the tests didn't, and it shipped anyway. That's not an AI problem — it's the same coverage gap teams have always had, just wider.
Most test suites still verify behavior against stubs and mocks — fake APIs, fake inboxes, fake state. When a human writes a login flow, they have context about what the real flow looks like. When an AI writes it, it's pattern-matching against training data. The resulting code might call the right endpoints in the wrong order, set a cookie the app never reads, or silently drop a header that a downstream service needs.
A stubbed test passes either way. A test that runs against the real app, through the real network, catches the difference.
This is the same argument we've been making since we built Test Chain as a proxy instead of an SDK — but it used to be a philosophical preference. Now it's a practical necessity. If 40–62% of AI-generated code ships with security or design flaws, the test that matters is the one that exercises the actual deployed behavior, not a mock of what we hope the behavior is.
The teams handling this well aren't doing anything exotic. They're doing the obvious thing harder:
None of this requires new tools or a new testing philosophy. It requires taking the testing you're already doing and making sure it actually touches the thing your users touch — not a sanitized version of it.
AI is making the code easier to write. It's not making the code easier to trust. That's still your job, and it's a bigger job than it was last year.