A test suite dies of a thousand small moves. Nobody breaks a feature. Someone renames a class, moves a button into a new container, changes “Sign in” to “Log in”, and each of those fails a test that was checking something else entirely. After a few months the suite is red often enough that nobody reads it, which is the moment it stops being a test suite.

This work is two engines that deal with that. One repairs the locator so the test can continue. The other decides whether what changed on screen was actually a regression.

Re-anchoring a locator

A locator is a path through the element tree. When the tree changes the old path no longer matches, but most of it is still right: the text is the same, the role is the same, only the wrapper moved. The healing layer treats the old locator and each candidate on the new page as sequences and scores them with a weighted variant of longest common subsequence. Attributes that are stable in practice, such as visible text, role and accessibility id, weigh more than ones that churn, such as generated class names and positional indices. The best candidate above a threshold becomes the new anchor, the test continues, and the repair is recorded for someone to accept or reject.

The weighting matters more than the algorithm. Plain LCS treats a matching index as equal to matching text, which is exactly backwards for a user interface: the index is the first thing to change and the text the last. Once the weights reflected how interfaces actually change, manual maintenance on the affected suites dropped significantly.

Visual regression without false positives

Pixel comparison on its own is a false-positive machine. Anti-aliasing differs between runs, the status bar shows a different time, a carousel is on a different slide. The engine layers three techniques so that each catches what the others miss. Pixel-level comparison handles regions that must be exact. Template matching finds where a component went, not only that it moved, so a relocated element is reported as relocated rather than as two failures. OCR-based text extraction compares text as text: a font-hinting difference is not a regression, a changed label is.

The change that removed most of the noise was region-scoped comparison. Compare what the test cares about and mask what it does not. A test for the checkout total should not fail because the header carousel rotated.

When to heal and when to fail

This is the difficult decision, and getting it wrong in either direction makes the tool worthless. Heal everything and the suite hides real regressions: a button that moved off-screen gets “found” and the run goes green. Heal nothing and you are back to a thousand small moves. The rules that held up: heal when the target is found with high confidence and the action then succeeds; fail loudly when confidence is low; and always surface what was healed, because a suite that silently repairs itself is as untrustworthy as one that fails constantly.

From rules to an agent

The later form of this idea lives inside CoTester as adaptive auto-healing. When a step fails, the agent analyses the failing element and its metadata to find the root cause, then generates replacement elements and corrective steps on the fly rather than picking from a fixed matching strategy. Same principle, different machinery. The rule-based version was what taught us which signals the agent needed to reason about, and the confidence thresholds transferred almost unchanged.

What it taught

  • Weight by stability, not by position. Text and role survive refactors; indices and generated classes do not.
  • Compare text as text. Pixels are the wrong representation for a label.
  • Scope the comparison. Most false positives come from comparing things the test never cared about.
  • Report every repair. Healing that nobody can see is a regression waiting to be missed.