Is Your Course Evaluation Survey Accessible? The Silent Non-Response Bias You Are Building In

Bottom line up front: A course evaluation that is hard to complete with a screen reader, a keyboard, or a cognitively overloaded mind does not merely inconvenience some students — it systematically removes them from your data. That makes accessibility not a side issue of web design but a core measurement problem: an inaccessible survey manufactures non-response bias against disabled and neurodivergent students, who are a large and growing share of the European student body. Under the EU Web Accessibility Directive it is also a legal obligation, not an aspiration.

Accessibility is a measurement problem, not just a compliance one

Quality assurance lives or dies on whether the people who respond resemble the people who were taught. The literature on non-response bias is unambiguous that the danger is not low numbers as such but systematic absence — when those who do not respond differ from those who do in ways that matter. An inaccessible instrument is a near-perfect engine for exactly that: it does not lose a random slice of students, it loses a specific slice, defined by disability.

And that slice is not small. EUROSTUDENT, the pan-European survey of student social conditions, found in its seventh round that roughly one in seven students across participating countries reported an impairment, long-standing health problem, or functional limitation affecting their studies, with mental-health conditions the most commonly reported — and later rounds, using broader definitions, report substantially higher figures (see the EUROSTUDENT project). These are precisely the students whose experience of teaching, support and assessment a quality process most needs to capture. If your evaluation form is the one place their voice goes missing, your "representative" data is quietly unrepresentative.

What actually breaks for whom

Inaccessibility is rarely dramatic; it is a thousand small frictions:

• Screen-reader users hit matrix/grid Likert questions that are not properly labelled, so the relationship between row and column is lost and the item becomes unanswerable.
• Keyboard-only users (including many with motor impairments) cannot reach a custom slider or star widget that assumes a mouse.
• Low-vision users face insufficient colour contrast or text that cannot be resized without breaking the layout.
• Neurodivergent students — those with ADHD, autism or dyslexia — disproportionately abandon long, dense grids of reverse-worded items, dropping out partway and contributing only partial, skewed data.

Each of these is a documented WCAG failure, and each one silently shifts your sample.

In the EU, this is now the law

For public universities, accessibility stopped being optional some years ago. The Web Accessibility Directive (EU) 2016/2102 requires public-sector bodies — which includes most state universities — to make their websites and mobile applications accessible, conforming to the harmonised European standard EN 301 549, which for web content maps to WCAG 2.1 Level AA. New public-sector websites had to comply from September 2019, existing sites from September 2020, and mobile apps from June 2021, with member-state monitoring bodies auditing conformance and an accessibility statement required.

The landscape has since widened. The European Accessibility Act (Directive (EU) 2019/882) became applicable on 28 June 2025, extending accessibility requirements to a broad range of private-sector products and services across the Union. Any evaluation tool a university procures — and any vendor selling one — now operates in an environment where WCAG-level accessibility is a baseline legal expectation, not a nice-to-have. An evaluation platform that cannot demonstrate conformance is a procurement risk as much as an ethical one.

But isn't this just a niche edge case?

The strongest objection runs: response rates are already low, most students are not disabled, and chasing WCAG conformance is gold-plating that will not move the aggregate numbers. Why over-invest for a minority?

Three answers. First, the premise understates the population: one in seven (and rising) is not an edge case, and that figure excludes the many students with temporary or situational limitations — a broken wrist, a migraine, a noisy environment, a phone with a cracked screen — whom accessible design also helps. Second, the aggregate argument is exactly the trap: a bias does not have to be large in volume to be large in consequence. If the students excluded are disproportionately those struggling with inaccessible teaching, your evaluation will systematically under-report the very problems it exists to surface, and it will do so invisibly. Third, accessible design is not a minority accommodation bolted on at cost; the "curb-cut effect" is well established — captioned media, clear language, keyboard navigation and uncluttered layouts raise completion and data quality for everyone, not only disabled users.

Designing evaluations that include everyone

Good practice is concrete and mostly free of heroics: use properly labelled, standards-compliant form controls instead of bespoke widgets; avoid dense matrix grids in favour of one clear question at a time; keep language plain and items short; ensure full keyboard operability and sufficient contrast; and test with assistive technology, not just with sighted mouse users. Crucially, treat accessibility as part of instrument validity — an item only measures what you think it measures if every intended respondent can actually answer it.

Procurement: the questions to ask a vendor

Because accessibility is now a legal baseline, the most efficient point of leverage is procurement. An evaluation tool that fails WCAG does not just risk a complaint; it imports non-response bias into every survey an institution runs on it. Quality and IT teams evaluating a platform should ask for evidence, not assurances:

• Is there a current accessibility conformance report? A VPAT or equivalent mapped to WCAG 2.1 AA / EN 301 549, dated and version-specific, not a marketing claim of being "accessible".
• Has it been tested with real assistive technology? Screen readers (NVDA, JAWS, VoiceOver), keyboard-only navigation and voice control — by users, not just automated scanners, which catch only a fraction of barriers.
• What is the survey respondent experience, not just the admin dashboard? Vendors often make the staff-facing tool accessible and neglect the student-facing form, which is precisely where the data is gathered.
• How are complex question types handled? Matrix grids, sliders and ranking items are the usual failure points; ask to see how each behaves with a screen reader.
• Is there an accessibility statement and a remediation process? The Web Accessibility Directive expects both; a vendor without them is one you will be remediating yourself.

Treating these as pass/fail procurement criteria does more for inclusive evaluation than any post-hoc fix. It also protects the institution: under the directive, the public body remains responsible for the accessibility of the services it deploys, regardless of who built them. The cheapest accessible survey is the one you never had to retrofit because you refused to buy an inaccessible one.

Where Koji fits

This is an area where a modern, conversational approach has a structural advantage. Koji for Education replaces the dense Likert grid — the single most common accessibility failure point — with an AI-moderated conversational interview that presents one question at a time, in plain language, and adapts to how the student responds. Its six structured question types (open-ended, scale, single- and multiple-choice, ranking, yes/no) are designed for clear, sequential interaction rather than a wall of matrix items, which lowers cognitive load for neurodivergent students and simplifies screen-reader navigation. Because the interview probes conversationally, a student who would have abandoned a twenty-item grid can give richer feedback in a format that meets them where they are, and Koji's GDPR/AVG-compliant, EU-appropriate data handling fits the same regulatory environment that the Web Accessibility Directive defines. The same conversational engine powers inclusive research on the main Koji platform for teams doing broader user studies.

Koji does not claim that conversational design eliminates every accessibility barrier — conformance still has to be designed, tested and maintained — but moving away from the matrix-grid form factor removes a large class of the failures that quietly bias today's evaluation data.

Stop deleting voices by accident. Explore Koji for Education to run evaluations every student can actually complete.