From test execution to release confidence

Context

A product organisation with several autonomous squads, one central group holding most of the end-to-end test coverage, and a release cadence that everyone wanted to be faster. Every squad shipped its own services. Nobody could say, on a given afternoon, whether the current build was safe to release, without asking the quality engineers first.

The constraint

The bottleneck was not the test suite. It was that release confidence lived in people rather than in the system.

Decisions queued behind a small group, because a squad could not answer a risk question without a handoff. That group became the escalation path for failures in code it had not written. And because no squad owned a quality signal, no squad had a reason to improve one.

That is an ownership failure that presents as a coverage problem. Adding tests would have made it worse, because it would have added more signal to the same narrow channel.

My role and decision scope

I led the quality engineering function: the strategy, the team, and the standard that other teams were asked to meet. I did not own the squads’ code and had no authority to change their priorities directly. Anything durable had to be agreed rather than imposed, which shaped the whole approach.

What I could decide: where my engineers spent their time, what the quality standard said, what evidence a release needed, and which of those things were negotiable.

What changed in the system

1. 01

   A central quality dependency

   One group holds the coverage and answers the release question on everyone else’s behalf.

2. 02

   Quality capability inside each squad

   Quality engineers embed with the people writing the code, early enough to shape a design rather than inspect it.

3. 03

   Squads own the release decision

   The team that owns the code states whether its own change is safe, against criteria it did not invent alone.

4. 04

   Shared standards and pipeline evidence

   What stays central: the agreed standard, the frameworks, and a pipeline that supplies the evidence behind the answer.

Quality engineers moved into the squads. Embedded rather than central, sitting with the people writing the code. The point was not proximity for its own sake. It was that a quality engineer inside a squad can influence a design decision before the code exists, which is the only cheap moment to influence it.

Expectations were written down and agreed, not announced. A common standard covering what a squad tests, at which level, and what evidence a release requires. Written collaboratively and taken through governance approval, because a standard that arrives as an edict gets satisfied on paper and ignored in practice.

Release confidence became a squad output. Each squad became responsible for stating whether its own change was safe, against shared criteria, with the pipeline supplying the evidence. Quality engineering owned the criteria and the tooling, not the verdict.

The central group’s job was redefined as capability. Frameworks, pipeline design, diagnosis of hard failures, coaching. The measure of success stopped being tests written and became squads that no longer needed us to answer a release question.

Evidence and outcome

The honest version: the durable evidence here is structural, not numerical. The standard exists, was approved through governance, and applies organisation-wide. Quality engineers are embedded in product squads rather than pooled centrally. Release decisions are made by the teams that own the code.

I have deliberately not published throughput or defect-rate figures. They are internal, and they depend on a baseline and a scope that a reader here cannot see. Where the numbers matter, I am glad to discuss them in a conversation where I can also say what they exclude.

What I would change

I under-invested in the diagnosis layer early on. Handing a squad ownership of a quality signal is only fair if the signal is legible: a failing test has to tell its owner what broke without a specialist translating. Where the failure output was poor, ownership transferred slowly and grudgingly, and squads quietly routed around it. Next time I would treat failure legibility as a precondition for devolving ownership, not as a follow-up task.

The second thing: I framed the standard as a standard. Framing it as a set of defaults a squad may deviate from, provided it says why, would have bought the same consistency with less resistance.