Skip to content
wcagc
←Back to blog
automated testingmanual testingWCAGEAA

Manual vs automated accessibility testing

Automated scans catch the machine-checkable issues; manual testing catches the rest. When you need each, and how to combine them in practice.

P

Pavel Charkasau

Manual and automated accessibility testing answer two different questions, so a serious program runs both. Automated testing checks whether the code meets a rule a machine can verify: is there an alt attribute, does the text hit the contrast ratio, does the form field have a label. Manual testing checks whether a person can actually use the page: can you reach the menu with a keyboard, does a screen reader announce the error, does the checkout finish. You need automation for speed and coverage on every deploy; you need manual review for the judgment calls a rule can't encode. The W3C is direct about the split — "no tool alone can determine if a site meets accessibility standards. Knowledgeable human evaluation is required to determine if a site is accessible" (W3C WAI). This guide covers what each method is good at, which WCAG checks fall on each side, and how to sequence them so you're not paying a human to find contrast errors a scanner would have caught in a second.

What's the difference between manual and automated accessibility testing?

Automation reads structure. A human reads meaning.

An automated scanner parses your rendered HTML, CSS, and ARIA and flags any failure it can confirm from the code alone. It runs in seconds, it runs the same way every time, and it scales to every page you point it at. That makes it perfect for regression testing — the same 200 pages, every deploy, no fatigue.

Manual testing puts a person in front of the page with a keyboard, a screen reader, and knowledge of how the standard works. They tab through the interface, listen to what NVDA or VoiceOver announces, zoom to 400%, and try to complete the actual task. They catch the things that turn on interpretation: whether the alt text describes the image, whether the reading order matches the visual order, whether a custom dropdown is operable at all.

Neither replaces the other. The W3C's evaluation methodology, WCAG-EM, assumes both: it is "an approach for determining how well a digital product conforms to WCAG," and it notes you'll "probably also want to use evaluation tools to be more efficient" while still requiring expertise in assistive technologies and how people with disabilities use the web (W3C WAI). The tool makes the human faster. It does not make the human optional.

When do you need automated testing?

Whenever the check is objective and the same failure could reappear on the next commit.

Automation earns its place on the repetitive, machine-decidable work:

  • Contrast ratios between text and background, measured against the WCAG threshold.
  • Missing alt attributes on images (presence, not quality).
  • Form fields with no programmatic label, where a screen reader reads "edit text" and nothing more.
  • Empty links and buttons — an <a> or <button> with no accessible name.
  • Missing page language (<html lang>), which decides how a screen reader pronounces the page.
  • Some ARIA misuse, like a role that requires a child element that isn't there.

These are common and cheap to fix once located, and they dominate the real web. In the WebAIM Million 2026 run across a million home pages, six error types accounted for 96% of all detected errors: low-contrast text (83.9% of pages), missing alternative text (53.1%), missing form labels (51%), empty links (46.3%), empty buttons (30.6%), and missing document language (13.5%) (WebAIM). Every one is machine-detectable. So automation isn't a token first pass — it clears most of the volume of failures before a human spends a minute. Wire it into CI and it catches the reintroduced contrast bug the day it lands, not at the next quarterly audit. That's the work to put behind a WCAG checklist and run on schedule.

When do you need manual testing?

Whenever the answer depends on meaning, sequence, or whether a task actually completes.

A scanner confirms an image has alt text; it can't tell you "chart-final-v2.png" is a useless description of a revenue graph. It confirms headings exist; it can't tell you the <h3> should have been an <h2>, or that the focus order jumps from the logo to the footer and skips the whole form. It can't open your menu with a keyboard, escape your modal, or hear that the payment error announces nothing to a screen reader. It won't know your error message says "invalid input" without naming the field. None of that lives in a rule a machine evaluates, and all of it is where real users get stuck.

Manual testing is also the only way to check a flow that needs a login. Most scanners see the marketing homepage; the barriers live in the authenticated pages behind it — the onboarding wizard, the cart, the account settings. Here's the opinion I'll stand behind: a green automated report on your homepage tells you almost nothing about whether someone can finish a purchase. The gap between "the scan is clean" and "a keyboard user checked out" is the whole game, and a person with a screen reader is who closes it.

Which WCAG criteria can a machine actually check?

Only a subset, and the honest numbers depend on what you count.

Count by WCAG success criteria: WCAG 2.2 AA has 55 of them, and a machine can evaluate with confidence only the ones that turn on code, not on judgment. That lands you in the widely quoted 20–30% range — the benchmark the field used for years. Count by volume of individual issues instead, and the number rises, because a few automatable failures are so common. Deque's analysis of more than 2,000 audits (roughly 13,000 pages, nearly 300,000 issues, all via axe-core) found automated testing fully covered about 57% of the issues found (Deque).

Both figures are true and they answer different questions. Neither one lets automation off the hook for the rest. The W3C's guidance on selecting tools states it plainly: "Tools cannot check all accessibility aspects automatically. Human judgement is required. Sometimes evaluation tools can produce false or misleading results" (W3C WAI). A tool can even flag a false positive (an ARIA pattern it misreads) that a human then has to overrule. The practical takeaway is not a number. It is that the criteria a machine skips (keyboard operability, meaningful alt text, reading order, screen-reader flows) decide whether the page works.

How do you combine them without wasting effort?

Scan first, fix what it finds, then put a human on what's left. In that order.

Running manual testing before automation wastes your most expensive resource. Don't pay an expert to find 40 contrast failures and 12 unlabeled inputs by hand when a scan surfaces them in seconds. The sequence that works:

  1. Automate the machine-checkable checks on every deploy. This gives you a fast, repeatable baseline and stops regressions from shipping. It maps to the code-decidable rows of the standard.
  2. Fix those in code. Contrast, labels, names, lang, obvious ARIA errors — clear the volume.
  3. Manually test the key flows with a keyboard and a screen reader: the top tasks, the authenticated journeys, anything custom. This is where WCAG-EM's structured, sampled evaluation belongs (W3C WAI).
  4. Write down what you checked and what you didn't — the honest record, not a pass/fail badge.

The two layers don't overlap. Automation is wide and shallow: every page, the same checks, forever. Manual review is narrow and deep: a few critical journeys, examined by someone who knows what a broken focus order sounds like. Skip the scan and you burn expert hours on trivia. Skip the manual pass and you ship a site that clears every scan and still can't be used.

What does this mean for EAA and EN 301 549?

It means a scan is necessary but not sufficient for what the law asks you to document. Under the European Accessibility Act, in-scope products and services have had to meet WCAG-level requirements since 28 June 2025, with the technical bar set by EN 301 549, which incorporates WCAG 2.1 Level AA for the web. EN 301 549 Annex C lists each requirement so you can mark it met, partially met, or not met — and "partially met" is the honest entry when automation covered a criterion but manual testing hasn't confirmed the rest.

The accessibility statement the EAA expects documents your level of conformance and known limitations, not a guarantee. That's why the manual/automated split isn't a technical footnote — it's the difference between a statement you can defend and one you can't. Claiming conformance off a clean scan alone is precisely the overreach that cost an overlay vendor $1 million at the US Federal Trade Commission (FTC). Scan to find the machine-detectable issues, test the rest by hand, and write down honestly where you stand.

Frequently asked questions

Is automated or manual accessibility testing better?

Neither — they do different jobs. Automated testing is faster, repeatable, and scales to every page, so it's better for catching machine-checkable failures on every deploy. Manual testing is better for keyboard operability, screen-reader flows, meaningful alt text, and whether a task completes. The W3C states that "no tool alone can determine if a site meets accessibility standards" (W3C WAI), so a real program uses both.

What percentage of accessibility issues can automated testing catch?

Roughly 20–30% of WCAG success criteria, or about 57% of issues by volume — the figures answer different questions. Deque's study of 2,000+ audits found automated testing covered about 57% of issues found, because a few machine-detectable failures are very common (Deque). The rest needs manual review.

Which accessibility tests must be done manually?

Keyboard operability, screen-reader announcements, reading and focus order, whether alt text is meaningful, whether custom widgets work, and whether authenticated flows like checkout can be completed. These depend on human judgment and can't be confirmed from code alone (W3C WAI).

Should I run automated or manual testing first?

Automated first. A scan surfaces the common machine-checkable failures (contrast, labels, empty controls) in seconds, so you fix those in code before spending expert time. Then a person manually tests the key flows with a keyboard and screen reader for the issues automation can't reach.

Does a passing automated scan mean my site is compliant?

No. A clean scan means the machine-checkable issues are handled; it says nothing about the criteria a machine never tested. Full conformance under the EAA and EN 301 549 needs manual testing plus an accessibility statement documenting your level of conformance and known limitations.

Get the automated baseline, then test the rest by hand

The workflow is unglamorous and it holds up: automate the machine-checkable half, then put a person on the half a machine can't see. Run a free scan to get the machine-detectable issues on your key pages, mapped to WCAG. Fix those in code, work through the manual criteria with a keyboard and a screen reader, and you'll have a real baseline — plus the evidence to write an accessibility statement that says what you actually checked.

Pavel Charkasau, founder, wcagc.com. Last updated 25 July 2026.

Sources