Software Engineering UKSoftware Engineering UK
How should Dev and QA align in the AI era?
Software Engineering UK

How should Dev and QA align in the AI era?

Omar EvansWords by Omar Evans

Further up the stream

Treat a change as unproven until somebody other than its author can say what else in production depends on the lines it touched. That is the test to apply before debating tooling, role titles or process, because it names the thing that has actually changed. Writing code is no longer the scarce step in delivery. Producing evidence that the code is safe to release is, and that evidence is manufactured at the boundary between development and quality assurance.

CIO reports a case that matters because of how ordinary it was. A minor edit to a shared component cleared review and QA. The tests passed, and the ticket's stated behaviour was delivered. The failure appeared elsewhere: the background step that converts a paid order into access stopped working, and a production log alert found it. The dependency list was absent, and so was the order in which the consumers had to move. The tests had been written from the same interpretation of the ticket as the implementation, so they could only agree with it.

Under AI the problem multiplies: generators produce small, plausible changes to shared surfaces faster than anyone documents those surfaces. That multiplication produces a class of failure that more unit testing cannot remove, because the missing information sits outside the diff and outside the test.

The bottleneck moved from typing to proving

The constraint on delivery has shifted to verification, and verification is exactly where the two roles meet. The Home Office's engineering standard for AI use is unusually direct about the consequence. It treats AI as an instrument and makes the human team answerable for anything the tool produces. It requires a visible trail in the normal engineering record, so a reviewer can reconstruct how a generated change was checked. It also asks teams to assume the output may be wrong, unsafe, partial or deceptive, and to accept that the symptom can surface long after the merge. That obligation is the design brief for the handoff.

The coding side of the bargain is to leave a change in a state that a different engineer can confirm. QA's half is to confirm it against something other than the change itself. InfoWorld's analysis of AI-generated code and engineering standards makes the enforcement point well: a rule needs to be written down, within reach and difficult to sidestep. Non-functional requirements in particular belong in the pipeline as acceptance checks, because engineers and generators alike chase a feature that runs and leave observability, error handling and audit trails for later. A standard that lives in a wiki lacks enforcement. A standard expressed as a check on the merge enforces itself.

Accountability does not move with the tool. If a reviewer cannot explain why a generated dependency was admitted, the review did not happen, whatever the record says.

The contract defines the evidence

In many teams the handoff is still a status change on a ticket, and that thin convention explains why it fails. A status transition carries no information. What the next stage needs is a small, fixed set of declarations that let it decide without reconstructing the author's context.

These carry most of the weight:

  • A pass-or-fail description of intended behaviour, settled before implementation, so QA tests the requirement itself.
  • A footprint of the change as built: every shared service redeployed, every interface altered, every data contract shifted, whether or not the ticket mentioned it.
  • Open questions and unfinished agent work, written into the record so the next stage knows what is still moving.
  • The proof each gate demands, and the name of the person allowed to carry whatever risk is left.

The second item deserves emphasis because it is where AI does its damage. QA should be comparing two maps: the surface the ticket asked for and the surface the change actually moved. When the second is larger, the change is incomplete until the difference is explained, and every extra service widens the test plan. The comparison is only possible if development declares its footprint honestly, which is why the declaration belongs in the contract.

Undeclared footprint is also how technical debt compounds. A change that arrives without a dependency map raises this release's risk and borrows against the next one, because whoever touches that component later inherits an undocumented set of consumers and no record of the order in which they must move.

The obligation runs both ways. QA owes development a signal that tells a product fault apart from a harness or environment fault; vague defect reports waste engineering time. QA also owes a clear list of missing proof and a note on which parts of the handoff worked, so the contract and the automation rules improve on the next pass.

A handoff record makes the contract concrete. This is the shape to start from, and the four evidence states matter more than the field names:

change: SUB-4412
acceptance: [AC-1, AC-2, AC-3]
touched:
  components: [billing/subscription]
  dependent_services: [entitlements, provisioning-worker]
  data_contracts: [subscription.v3]
generated_share: 0.6
agent_permissions: scoped
open_items:
  - provisioning-worker ordering unconfirmed
evidence:
  unit: passed
  contract: failed
  integration: blocked
  security: empty
risk_tier: moderate
sign_off: [dev-lead, qa-lead]

Blocked and empty are the two states that stop a green build from lying. A test that was started but could not finish is not a pass. A test that was never run is not a pass either. Recording both explicitly forces the absence of proof into the release decision, and a build badge cannot stand in for it.

Measure whether checking keeps up with generating

The useful question is how fast a team can reach a release call that proof supports. Take the proportion of proof items a risk tier demands that carry a definite pass or fail, then divide it by the clock time between ready-for-review and the logged decision. Apply the ratio only after that proportion reaches the tier's threshold; a quick call on half the proof is still a guess.

The threshold is where risk tiers earn their keep. Not every change deserves the same proof.

Risk tier What places a change there Evidence the handoff must carry Who accepts residual risk
Low New code in one service, narrow tool permissions, existing checks cover the surface Criteria-based unit tests, review record, declared file scope Reviewing engineer
Moderate Shared component, interface or data contract in play, or generated code forms much of the diff The above plus contract tests, integration checks, the two-map comparison and a declared dependency order Development and QA leads jointly
High Older system whose context must be rebuilt, or blast radius reaching customer-facing flows The above plus security and performance evidence, staged rollout, and a production signal tied to the change Architecture owner with product
Critical High tier plus multi-service blast radius, regulated data, or a security boundary The above plus independent security review, failure-injection evidence, and a rollback rehearsal Engineering leadership with service owner

Begin with one live delivery path. Put the contract on it, record the four evidence states, and compare its evidence completeness and validation lead time with its own baseline. Give the tools more latitude only when both measures move the right way and production stability holds. That is how the two roles align: the coder leaves proof that someone else can check, and the checker turns that proof into a release decision the business can defend.

Sources

the human team answerable for anything the tool produces
Home Office's engineering standard for AI use

Source: What developers and QA need from each other when AI writes the code, CIO

The usual questions

Why is unit testing insufficient for AI code?

AI generates changes to shared surfaces quickly, creating failures where missing information exists outside the diff and the test.

How should engineering standards be enforced?

Standards should be expressed as checks on the merge rather than in a wiki, as pipeline acceptance checks are harder to sidestep.

What should a Dev to QA handoff include?

It needs a description of intended behaviour, a footprint of the change, open questions, and the specific proof each gate demands.

Filed so far