Digital transformation baseline · Article 12

What Lighthouse Does Not Tell You

Every article in this series has said the same careful thing about its own accessibility score — that 100 out of 100 is a floor, not a certificate. This is the piece that finally pays for that sentence. Nineteen manual sessions on macOS across three browsers and six templates, VoiceOver throughout, plus a real-device pass on an iPhone and an Android handset. The automated number did not move, because it was already zero and stayed zero. A person found three things anyway, and none of the three are things a tool has a rule for.

Twelfth in the digital transformation baseline series.

0 Violations found by automated testing Every page, every build, for as long as the guardrail has existed. The number has never been anything else.
3 Findings from the manual pass Logged as A11Y-9, A11Y-10 and A11Y-11. Not one of them is a thing an automated rule could have fired on.
19 Manual sessions logged so far Six templates, three browsers, three engines, VoiceOver on macOS — plus a real iPhone and a real Android. Still not the whole pass.

A perfect automated accessibility score means the markup is well-formed. It does not mean the page is usable, and it has never claimed to. The tool this project runs says so about itself, in its own dependency note — axe catches roughly thirty to forty per cent of issues. This is an account of the other sixty.

What the automated pass actually checks

@axe-core/cli 4.10, against every built page, as part of the same command that blocks a deploy.

The number is genuinely zero, and it is worth being precise about what that buys. Every image has an alt attribute. Every form field has a programmatically associated label. Contrast ratios pass. There is one <h1> per page and no skipped heading levels. Landmarks are present and unique. No element carries an ARIA role it fails to honour. All of that is real, all of it is checked on every build, and a regression in any of it fails the run rather than reaching a person.

What every one of those checks has in common is that it can be answered by looking at the document alone. Does the attribute exist. Is the ratio above the threshold. Is the role valid for the element. A machine can answer those completely and tirelessly, which is exactly why it should — and why the score stays at zero without anybody thinking about it.

The questions a machine cannot answer are the ones that need a second document to compare against — the one in the reader's head. Does this label still mean what it means visually. Does the page tell you what just changed. Should this image be speaking at all.

What a person found

Nineteen sessions across Safari, Chrome and Firefox on macOS, VoiceOver throughout, plus a real-device pass on an iPhone and an Android handset. Three findings, each re-confirmed against the DOM before being written down.

  • A label that only existed if you could see it (A11Y-9, since fixed). The Collaboration page's services carry six eyebrow labels above their headings — "The Engine", "The Fuel", "The Runway", and so on. Each was a bare paragraph with no connection to the heading beneath it. A sighted reader saw a label attached to a service. Someone navigating by heading list never heard it at all, and reading straight through got the phrase as an orphan, floating before a heading it had no stated relationship to. Axe saw a valid paragraph followed by a valid heading; there was no rule to break. Each label now sits inside the heading it labels, so the accessible name carries both halves.
  • A section that closes behind you without saying so (A11Y-10). The FAQ items are mutually exclusive — opening one closes whichever was open. Open three questions in turn and exactly one is open, which the page never mentions. Each toggle reports its own state correctly, so the machine-checkable contract is kept in full; the thing that breaks is the reader's model of the page, and only if they are operating it without seeing it.
  • Eight photographs that describe themselves at length (A11Y-11). Every stock photo on the brochure pages announces a full sentence about the photograph. The alt text is well written — the tester's note says it read fine, which is the point. The question a tool cannot reach is whether an illustrative image carrying no information the surrounding copy lacks should be announced at all, or should be silent. Axe checks that alt exists. It has no way to ask whether it should.
Not failures of markup — failures of meaning

Read the three together and they are the same finding wearing three outfits. In each case the markup is correct, the machine-checkable contract is satisfied, and something true of the visual page is not true of the page a screen reader describes. That gap is invisible to a validator by construction, because a validator only ever has one of the two versions to look at.

The through-line so far

An automated pass proves the markup is well-formed. It cannot prove the markup is true.

Why the score was never the claim

None of this makes the automated pass decorative. Zero violations across every page on every build is the reason the manual session could be spent on meaning rather than re-checking whether an image had an alt attribute. The tool cleared the floor so a person could look at the room.

What it does mean is that the score was never the thing being claimed. Every previous article in this series carried a version of the same sentence — that 100 out of 100 is a floor rather than a certificate — and that sentence was, until this pass ran, a promise rather than a demonstrated fact. Three findings later it is a demonstrated fact, and the honest version of the headline is not "our site is accessible" but "our automated score is perfect and a person still found three things in an afternoon."

What this doesn't settle

  • The pass is substantial, not complete — and this bullet had to be corrected on publication day. It first claimed the sessions ran on iOS and that Android and TalkBack had not been touched. Both were wrong. The nineteen sessions ran on macOS, where Safari, Chrome and Firefox are WebKit, Blink and Gecko rather than three coats of paint on one engine, and both handsets had in fact been exercised — the iPhone with Speak Screen, the Android with TalkBack, across all six templates. The error understated the browser coverage and overstated the device coverage at the same time, which is a tidy demonstration of why a testing log is worth keeping precisely rather than approximately.
  • The Speak Screen gap named above closed the same day. A second, proper pass followed the smoke check — VoiceOver on the iPhone, TalkBack on the Android, across all seven templates including this series' own hub — and it is what section 2 actually asks for, not a substitute for it. It found two more things no automated tool has a rule for: the skip link doesn't appear in the viewport when reached by swiping to it on a real device, confirmed independently on both platforms, and the Reach Out form's honeypot field — deliberately built to announce "leave this field blank" to anyone who does land on it outside the tab order — does exactly that, with both testers agreeing it's still one extra stop before a required checkbox. Logged as `A11Y-12` and `A11Y-13`. Neither fixed yet.
  • One template was never in either pass. The article template these words are sitting in has not been through the script at all.
  • One of the three is fixed. Two are still open. A11Y-9 was closed the same day this piece went up — each label now sits inside the heading it labels, so the accessible name carries both halves where it previously carried only the second. The other two are open on purpose rather than by neglect. The FAQ behaviour is a decision to make deliberately, not a defect to patch, and the photograph descriptions need re-confirming image by image rather than a blanket rule applied to all eight.
  • "Not obvious" is what most of the log actually says. Eighteen of the nineteen rows record no obvious failure, which is a genuine result and not the same result as "nothing is there." A pass that finds little on a page built carefully is the expected outcome; it is not proof that a different reader, on a different device, with different needs, would find the same.

The through-line

The most useful thing the manual pass produced was not the three findings. It was the demonstration that a perfect automated score and three real problems can be true of the same page at the same time, and that only one of those two facts shows up on a dashboard. Every article here has said the score is a floor. This is the one that went and stood on it.