How to Choose Course Evaluation Software: RFP & Requirements Guide (2026)

Most "best course evaluation software" lists rank products. A procurement process ranks requirements — then scores products against them. This guide gives QA directors, heads of teaching and learning, institutional-research leads, and procurement teams a defensible, audit-friendly way to run that process: eight weighted requirement categories, a scoring matrix you can drop into an RFP, a question bank vendors rarely volunteer answers to, and a short demo script that surfaces the differences that matter.

The short answer: score every shortlisted tool against eight categories — pedagogical fit, qualitative depth, response rates, data protection and hosting, accessibility, LMS/SIS interoperability, closing-the-loop workflow, and total cost — weighted for your institution. The winner is the tool that scores highest on your weights, not the one with the longest feature list. For most European universities in 2026 the sharpest differentiators are qualitative depth, GDPR/hosting posture, and whether the tool actually helps you act on feedback rather than just collect it.

Step 1: Write requirements before you look at products

The single most common procurement mistake is demoing tools first and reverse-engineering requirements from what you saw. That lets vendors set your criteria. Instead, agree the eight categories below with academic stakeholders, your DPO, and your accessibility lead before the first demo, and assign each a weight from 1 (nice to have) to 5 (must have) for your context. A research-intensive university with 40% international students will weight multilingual qualitative analysis and doctoral-cohort reporting far higher than a small teaching college will.

Step 2: The eight requirement categories

1. Pedagogical fit. Does the instrument support the questions you actually ask — formative mid-module check-ins as well as summative end-of-term evaluation? Can academics tailor items without breaking cross-cohort comparability? Static Likert-only tools force a trade-off between standardisation and usefulness; conversational tools can standardise the frame while probing individual answers.

2. Qualitative depth. Open-text comments are where the actionable insight lives, and where legacy tools are weakest. Ask how the tool handles free text: manual reading, keyword tagging, or genuine thematic analysis? Does it probe vague answers ("the pace was off") for specifics, or just record them? This is the category where AI-moderated interviews (Koji) and text-analytics add-ons (Explorance MLY, Qualtrics Text iQ) separate from form builders.

3. Response rates and representativeness. Low, skewed response rates are the number-one threat to the validity of your evidence — and to how accreditors read it. Ask about mobile completion, reminder automation, LMS-embedded launch, and reporting that shows who responded, not just how many. Higher engagement from a conversational format only helps if the tool also lets you check representativeness.

4. Data protection and hosting. For European institutions this is often decisive. Confirm EU/EEA data residency, a GDPR-compliant Data Processing Agreement, sub-processor transparency, and whether a Data Protection Impact Assessment (DPIA) is required under GDPR Article 35. Ask where data actually lives post-Schrems II and whether the vendor holds ISO 27001 or equivalent. Since 2 August 2026, obligations under the EU AI Act also apply to certain AI systems — ask any AI-enabled vendor how they classify their product and document it.

5. Accessibility. A student-facing survey is, for many institutions, a legal accessibility obligation. The presumed EU compliance standard is EN 301 549, whose current version (V3.2.1, 2021) incorporates WCAG 2.1 Level AA; version 4.1.1 (expected 2026) moves to WCAG 2.2. The European Accessibility Act became enforceable on 28 June 2025 for many private-sector services. Ask for a current VPAT/accessibility conformance report — not a marketing claim.

6. Interoperability (LMS/SIS). Manual roster uploads are error-prone and a data-protection risk. The vendor-neutral standard is 1EdTech's LTI 1.3 (with LTI Advantage services: Names and Role Provisioning, Deep Linking, and Assignment and Grade Services), built on OAuth2, OpenID Connect and JWT. Ask whether the tool is 1EdTech-certified and how it syncs with your SIS (Banner, SITS, etc.) for enrolment-driven distribution.

7. Closing-the-loop workflow. Collecting feedback is the easy half. Accreditors and students both want evidence you acted. Ask whether the tool tracks actions from insight to change, supports longitudinal cohort reporting, and produces the standardised outputs your quality cycle and accreditation self-evaluation need.

8. Total cost of ownership. Look past the licence: implementation, LMS integration, training, support tier, translation costs, and the internal analyst hours legacy tools consume on manual coding. A cheaper licence that needs two FTE to run is not cheaper.

Step 3: The scoring matrix

Score each shortlisted tool 1–5 per category, multiply by your weight, and total. Publish the matrix in your business case — it makes the decision defensible to finance and to an audit.

Requirement category	Weight (1–5)	Tool A	Tool B	Koji	What "5" looks like
Pedagogical fit	_	_	_	_	Formative + summative; standardised yet tailorable
Qualitative depth	_	_	_	_	Automatic thematic analysis; probes vague answers
Response rate & representativeness	_	_	_	_	Mobile-first, LMS-embedded, respondent-profile reporting
Data protection & hosting	_	_	_	_	EU residency, DPA, ISO 27001, DPIA support
Accessibility	_	_	_	_	Current EN 301 549 / WCAG 2.1 AA conformance report
Interoperability (LTI/SIS)	_	_	_	_	1EdTech-certified LTI 1.3 + SIS sync
Closing-the-loop workflow	_	_	_	_	Action tracking + longitudinal reporting
Total cost of ownership	_	_	_	_	Transparent, low internal-labour overhead

Leave the tool columns blank in your RFP and let evidence — not the sales deck — fill them in.

Step 4: The RFP question bank

Paste these into your RFP; they surface the answers vendors don't volunteer:

In which country/region is our data stored and processed, and who are your sub-processors?
Will you sign our DPA, and can you support a DPIA under GDPR Article 35?
Do you hold ISO 27001 (or equivalent)? Please attach the certificate.
How does your product classify under the EU AI Act, and what documentation do you provide?
Provide your current EN 301 549 / WCAG 2.1 AA conformance report.
Are you 1EdTech-certified for LTI 1.3? Which LTI Advantage services are supported?
How do you analyse open-text responses — manual, keyword, or thematic — and in how many languages?
What reporting do you provide on who responded (representativeness), not just the count?
Show how an insight becomes a tracked action and appears in a closing-the-loop report.
Itemise all costs: licence, implementation, integration, translation, training, support.

Step 5: The demo and pilot script

Demos flatter every tool. A structured pilot doesn't. Run the same real module through each finalist and check: How long does setup take? Can an academic launch it without IT? What does a student see on a phone? Feed in 30 genuine open-text comments and compare the analysis. Then produce the exact report your next accreditation self-evaluation will need. The tool that makes that report easiest usually wins.

When a competitor — or a simpler tool — is the right call

Honesty converts better with this audience than overclaiming, so be clear about fit:

You only need a quick, one-off poll. Microsoft Forms or Google Forms are free and adequate. A dedicated platform is overkill. (See our Koji vs Microsoft Forms and Koji vs Google Forms comparisons.)
You have deep in-house survey-methods expertise and self-hosting capacity. LimeSurvey or SoSci Survey give you total control at low licence cost. See open-source and free options.
You are standardised on a large experience-management suite already. If Qualtrics is institution-wide, its course-evaluation module may be the path of least resistance — weigh that against qualitative depth (Qualtrics alternatives).
Your priority is a mature, paper-and-scanning central evaluation office. EvaSys remains strong on high-volume centralised administration (EvaSys alternatives).

Koji is the better choice when qualitative depth, standardised bias-aware moderation, multilingual analysis, GDPR/EU hosting, and closing-the-loop action tracking are what move your weighted score.

How Koji maps to the eight categories

Koji is an AI-native evaluation platform built on the same AI interview engine as the main Koji research platform (koji.so). Against the matrix, it targets: qualitative depth (AI-moderated interviews that probe for specifics and analyse themes automatically), response rates (conversational, mobile-first collection), standardisation (a consistent moderation frame across every student, reducing item-order and interviewer effects), data protection (EU/GDPR data handling and a DPA), closing the loop (action tracking and longitudinal cohort reporting), and multilingual analysis across cohorts. Score it on your weights like any other tool — the point of this guide is that the process, not the vendor, produces the decision.

Related resources

Ready to score Koji against your requirements?

Bring your weighted matrix to a Koji demo and we will complete the Koji column against evidence — DPA, hosting, accessibility report, and a live closing-the-loop example. Book a walkthrough and run the same pilot module you give every finalist.

Frequently Asked Questions

The questions below reflect what procurement and QA teams most often ask when scoping a course evaluation purchase.

What should be in a course evaluation software RFP? At minimum: data residency and GDPR/DPA terms, ISO 27001 status, EU AI Act classification, an EN 301 549 / WCAG 2.1 AA accessibility conformance report, 1EdTech LTI 1.3 certification and SIS sync, open-text analysis method and languages, representativeness reporting, closing-the-loop action tracking, and a fully itemised cost breakdown.

How do I compare course evaluation tools objectively? Write weighted requirements before demos, score each finalist 1–5 per category against evidence (not the sales deck), and run the same real module through every tool as a pilot. Publish the scoring matrix in your business case so the decision is auditable.

Is Qualtrics or EvaSys better for European universities? It depends on your weights. EvaSys is strong for high-volume centralised administration; Qualtrics fits institutions already standardised on its suite. Both are Likert-survey-centric, so they score lower where qualitative depth and automated thematic analysis carry heavy weight.

Do we need a DPIA for course evaluation software? Often, yes. If processing is likely to result in high risk to individuals — for example large-scale processing of student data or AI-based analysis — GDPR Article 35 requires a Data Protection Impact Assessment. Ask vendors whether they support the DPIA and can document their processing.

Does course evaluation software need to be accessible by law? For most EU institutions, yes. Student-facing surveys fall under the Web Accessibility Directive (public sector) and, since 28 June 2025, the European Accessibility Act for many services. The presumed standard is EN 301 549, incorporating WCAG 2.1 Level AA — request a current conformance report.

Where does Koji fit? Koji is strongest when qualitative depth, bias-aware standardised moderation, multilingual analysis, EU/GDPR hosting, and closing-the-loop action tracking dominate your weighted score. For quick one-off polls or pure self-hosting, a simpler or open-source tool may be the better call.