Last December, Forrester published its Q4 2025 Wave and quietly retired the old category name. "Continuous Automation Testing Platforms" became "Autonomous Testing Platforms." Fifteen vendors were evaluated. Press releases were issued. Leaders were crowned. The word "autonomous" appeared in more earnings calls in Q1 2026 than in the previous five years combined.
Meanwhile, the number that actually matters — the percentage of test cases that run without a human — hasn't moved. Forrester's own analysis shows most organizations plateaued at 25% automated coverage. That ceiling has held for years. It held through the Selenium era. It held through the Cypress era. It held through Playwright passing both of them. And it's holding through the autonomous era too.
The industry renamed the problem. The problem didn't notice.
Read the Wave criteria and a pattern emerges. Self-healing locators. AI-generated test scripts. Natural-language test authoring. Risk-based test selection. Every vendor in the evaluation — Tricentis, UiPath, Functionize, ACCELQ, Applitools, Keysight, LambdaTest, all fifteen — is building some version of the same thesis: the reason coverage is stuck at 25% is that writing and maintaining tests is too hard, and AI can make it easier.
That thesis is half right. Writing tests is hard. Maintaining them is harder — QA teams spend 40 to 60 percent of their time on test maintenance, not writing new tests. Every UI change, every API update, every design tweak sends a ripple through the suite that requires human intervention. Self-healing locators genuinely help with that. Forrester puts the maintenance cost reduction at 40 to 45%.
But making tests cheaper to write and maintain only matters if you know what to test. And that's the half the thesis misses.
The 25% plateau isn't caused by a shortage of people who can write test scripts. It's caused by a shortage of people who know which flows matter enough to test.
Think about what the other 75% contains. It's not the login page — everyone tests the login page. It's the flow where a user signs up via Google OAuth, gets redirected to an onboarding wizard, skips step 3, goes back to step 2, changes their email, hits a validation error, fixes it, and lands on a dashboard that loads data from three different APIs. Nobody writes that test because nobody thinks of that test. It's a path that exists only because a real user took it.
AI-generated tests have the same blind spot. An autonomous agent can look at your codebase and generate tests for the functions it can see. It can crawl your UI and generate tests for the pages it finds. What it can't do is generate a test for a flow that only exists in the gap between what the code does and what users actually do. The agent doesn't know about the user who always opens the settings page before checkout because they're checking their shipping address. It doesn't know about the flow that breaks only when you navigate from the pricing page because that page sets a cookie the checkout page doesn't expect.
The coverage ceiling is a knowledge problem. The system doesn't know what the real flows are, so it can't test them — no matter how autonomous the testing platform becomes.
There are two ways to get a test for a flow that nobody thought to write.
The first is to make AI smart enough to discover it. Train an agent on your app's state space, let it explore exhaustively, and hope it stumbles onto the path that matters. This is what the "autonomous" category is building toward. It's computationally expensive, probabilistically incomplete, and — based on the numbers — adding about 12% more coverage over manual approaches. Real, but modest. Not the ceiling-breaker the category promises.
The second is to record a real user doing it. A user navigates your app. Every click, every input, every redirect, every response header is captured. The flow exists because someone actually took it. The test exists because someone hit record.
The difference isn't philosophical. It's informational. A generated test is the AI's guess about what matters. A recorded test is evidence of what matters. One of these has a coverage ceiling built into its architecture. The other has a coverage ceiling determined by how many flows you bother to record.
The autonomous testing pitch leans hard on maintenance reduction, and for good reason. If your existing 25% of tests cost 40-60% of your QA time to maintain, adding more tests makes the maintenance problem worse, not better. The ceiling isn't just "we can't write more tests" — it's "we can't afford to maintain the tests we already have."
Self-healing helps. When a button ID changes from #submit-btn to #submit-button, a self-healing locator adapts automatically instead of failing. That's a real improvement over the Selenium-era brittleness that made test suites rot faster than teams could repair them.
But self-healing operates at the locator level. It fixes the selector. It doesn't fix the flow. When the app redesigns its checkout from a single page to a multi-step wizard, every test that interacted with the old checkout is structurally broken — not because a locator changed, but because the steps changed. No amount of healing fixes a test whose fundamental assumption about the flow is wrong.
Recording handles this differently. The old test is wrong, yes. But re-recording the new flow takes the same time as the first recording: walk through it once. There's no selector surgery, no step-by-step repair, no debugging why the healer chose the wrong element. The new flow is the new test.
Forrester's evaluation criteria include AI capabilities, integration breadth, platform scalability, and enterprise governance. All valid dimensions for comparing platforms. But the Wave doesn't measure the thing that determines whether any of these platforms actually move the 25% number: how much of your app's real user behavior is captured in your test suite.
You can have a Leader-quadrant autonomous testing platform with world-class self-healing, natural-language authoring, and risk-based test selection. If your tests don't cover the flows your users actually take, you're autonomously testing the wrong things.
The category got a new name. The vendors got new capabilities. The coverage ceiling held. Maybe the problem isn't how the tests are written, maintained, or selected. Maybe the problem is that nobody asked the users which flows matter — and recorded the answer.