What Playwright failures are actually telling you

Most Playwright failures are not defects. They are the suite reporting on its own architecture, and on how testable the application underneath it really is.

That distinction decides what happens next, which is why it is worth being strict about. A failure read as an isolated defect gets fixed where it appeared. A longer timeout on this test, a more specific selector on that one, a retry on the unreliable one. The suite goes green, the pull request merges, and the same class of failure arrives the following week in a different test and gets fixed in place again. Nobody involved is being careless. They are answering the question the failure appears to ask instead of the one it is actually asking.

Here is how I read the common ones.

A timeout on a locator

This one is almost never a slow application, and treating it as one is how suites acquire the long waits that make every later run expensive.

Two causes cover most of them. The first is a locator that is ambiguous or coupled to markup. It matched during development and stopped matching when a component was restructured, a wrapper element appeared, or a class name changed for reasons that had nothing to do with behaviour. The test is not asserting on what the user sees. It is asserting on how the page happens to be built this week.

The second is a state the test assumed rather than created. The element genuinely is not there, because the account is in the wrong state, the basket is empty, or a previous test left something behind. Raising the timeout gives the application more time to produce something it was never going to produce.

The useful response is to ask which of those two it is before touching a single number. If it is the locator, the fix belongs partly in the application: a stable role, an accessible name, a deliberate test id. If it is the state, the fix is that the test sets up its own preconditions instead of inheriting them.

A strict-mode violation

Playwright refusing to act because a locator matched more than one element is the most useful failure on this list, and it is the one most often defeated rather than read.

It found a real ambiguity. Either the page contains two things a person would also struggle to tell apart, which is worth knowing, or the locator is not describing the element with enough precision. Reaching for the first match resolves the error without resolving either problem, and it converts a loud failure into a quiet assumption about ordering that will hold until it does not.

Passes locally, fails in CI

The instinct is to blame the environment. Sometimes that is right. Read it as a parallelism or shared-state signal first, because that is the more common cause and the more expensive one to leave in place.

Tests that pass in isolation and fail together are usually competing for something: a shared account, a fixture built once for a whole module, a query that takes the most recent record, a value chosen at random from a pool that another worker is also drawing from. Locally the suite may run with less parallelism, or in an order that happens to be kind. CI removes that kindness. The environment did not break the test. It stopped hiding a dependency the test always had.

Passes on retry

This is the most expensive of the four, because it does not look like a failure at all. A green run after a retry closes the question. The reason it failed the first time is still there, is still unexplained, and will now be discovered by someone else, later, with less context and under more time pressure.

I am not against retries. They are a reasonable way to keep a pipeline moving while a known cause is being fixed. They stop being reasonable the moment they become the fix, because at that point the suite has quietly stopped reporting on the thing it exists to report on. If a test passes on retry, the retry buys time. It does not buy an answer.

Traces are the part worth investing in

Everything above depends on being able to see what actually happened, and a team that cannot do that will keep choosing the cheap explanation because it is the only one available.

A trace turns triage into a short, factual exercise. What the page looked like at the moment of failure, what the locator resolved to, what the network was doing, what ran immediately before. The difference between a minute of triage and an afternoon of it is almost entirely whether that evidence was captured on the failing run or has to be reconstructed by rerunning things and hoping.

Which is why I treat trace capture, readable failure output and a suite whose results can be trusted on the first read as part of the test infrastructure rather than as a convenience. I have written elsewhere about what that takes in practice: Browser automation a team can trust on the first read.

What this changes

Read one failure at a time and you get a queue of small fixes. Read the failures together and they describe something specific about how the suite is built and how testable the application is, which is a different and more useful conversation to bring to a product team.

The question I would ask of a suite is not how many tests it has or what its pass rate looks like this week. It is what its failures have been telling everyone, and whether anyone has been listening to that rather than to the individual tests.