Accessibility Conformance Testing (ACT) Explained

What is accessibility conformance testing?

Accessibility conformance testing is the process of checking a website, app or document against an accessibility standard, usually the Web Content Accessibility Guidelines (WCAG), and recording for each requirement whether it passes, fails or does not apply. The result is a list of findings against a stated conformance level (A, AA or AAA) that a team can fix and a buyer can read. The same words also name a W3C format, the ACT Rules Format, which describes how individual test rules are written.

Two meanings of "ACT"

• The activity. Evaluating a product against WCAG (WCAG 2.2 is the version W3C recommends) or a standard built on it, using tools and human review.
• The W3C ACT Rules Format. A W3C Recommendation (version 1.1) that defines a common structure for test rules: what the rule applies to, what it expects and one of five outcomes: passed, failed, inapplicable, cantTell or untested. Rules can be automated, manual or a mix, which lets different tools and testers interpret a requirement the same way.

How conformance testing works

• Define scope and target. Pick the product, the pages or flows in scope and the level, usually WCAG 2.2 AA.
• Choose a sample. When checking every page is not feasible, W3C's WCAG-EM method recommends a representative sample of views, selected in a structured and a random way.
• Run automated checks. Tools such as axe or Lighthouse find issues that can be detected from the code, for example missing form labels or low contrast.
• Do manual checks. People test keyboard navigation order, focus visibility and reading order, and use a screen reader. See manual tests for accessibility.
• Report. Record pass or fail per success criterion with the steps taken, then retest after fixes.

Example

A team targets WCAG 2.2 AA for a checkout flow. An automated scan flags text at a contrast ratio below the 4.5:1 that WCAG requires for normal text. A manual pass then finds that the coupon field cannot be reached by keyboard, which no scanner reported. Both go into the report as failures against named success criteria, and both are fixed and retested.

Automated checks cover only part of WCAG

Lighthouse's accessibility score is a weighted average of automated pass or fail audits, and its own documentation lists manual checks, such as logical tab order and focus traps, separately from the score. A perfect score therefore does not mean a page conforms to WCAG. Related: automated accessibility testing.

Conformance testing and the ACR

An Accessibility Conformance Report (ACR) is the completed VPAT template, and it should be based on a manual evaluation, not on a scanner run alone. The Information Technology Industry Council (ITI), which publishes the VPAT, does not review or approve VPATs and there is no VPAT certification, so the vendor is responsible for the accuracy of the report.

Common mistakes

• Treating a Lighthouse score of 100 as a conformance claim.
• Testing only the home page instead of a sample that includes forms, tables, dialogs and error states.
• Skipping assistive technology: testing only with a mouse hides most keyboard and screen reader failures.
• Not stating the standard, version and level, so nobody can tell what "compliant" means.

Where LoadFocus fits

LoadFocus does not produce accessibility conformance reports or certify conformance. Page speed monitoring runs scheduled Lighthouse audits that include an Accessibility score, which is useful as an automated early warning between full evaluations. For how performance and accessibility testing fit together, read why performance testing must include accessibility.

FAQ

Can automated tools prove WCAG conformance?

No. Automated tools detect only the issues that can be found from code, and Lighthouse itself lists manual checks separately from its score. Conformance needs a manual evaluation as well.

What is the difference between ACT and WCAG-EM?

The ACT Rules Format defines how individual test rules are written so that tools and testers get consistent results. WCAG-EM is a W3C method for evaluating a whole website: scope, sample, evaluate and report.

Do I need conformance testing to get an ACR?

Yes. An ACR states how a product meets each criterion, and it should rest on an evaluation of that product. ITI does not review ACRs, so accuracy is the vendor's responsibility.

Which WCAG version and level should I test against?

W3C recommends the latest version, WCAG 2.2. Level AA is the usual target for procurement and legal requirements, but check the standard your buyer or regulator names.