Institution-Level Evaluation Reporting for Quality Audits: Building Longitudinal, Cross-Programme Evidence

Answer first: what an institutional audit actually needs from your evaluation data

When a quality-assurance director prepares for an institutional review or audit — as distinct from a single programme accreditation — reviewers are not asking whether one module ran a survey. They are asking whether the institution has a system: does it systematically collect student feedback, analyse it, act on it, and demonstrate that the loop closes, consistently, across every programme, over several years?

That is a fundamentally different evidence problem from programme-level accreditation. Programme accreditation asks "show me this programme." An institutional audit asks "show me that your internal quality system works everywhere, reliably, and improves things." Under the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG 2015), Standard 2.1 expects external review to address the effectiveness of the internal quality assurance described in Part 1 — so the audit is, in effect, a test of your internal system, not just its outputs.

This guide maps the institutional-audit evidence requirements to concrete outputs, and shows where a modern evaluation platform such as Koji produces them directly — and, honestly, where a traditional survey system plus an institutional-research team may still be the right combination.

Programme-level vs institution-level evidence

Most course-evaluation tooling is built to report at the level of a module or a programme. That is necessary but not sufficient for an institutional audit. According to EQAR, of the 8,929 external reviews carried out by its 46 registered agencies in 2018, roughly 91.8% were at programme level and only 7.8% at institutional level — institutional review is rarer, higher-stakes, and evidentially more demanding, because it must aggregate.

Institution-level reporting requires three things that per-module reporting usually cannot deliver on its own:

1. Aggregation across programmes and faculties — a roll-up that shows the whole institution, not a folder of separate PDFs.
2. Longitudinal trends — the same measures tracked across cohorts and years, so reviewers can see improvement (or its absence) over time.
3. Consistency and comparability — evidence that every unit is asked in a broadly standardized way, so cross-unit comparison is meaningful rather than an artefact of different survey designs.

If each department runs its own differently-worded survey, the institution cannot honestly aggregate the results, and reviewers know it. Standardization is not bureaucratic tidiness; it is what makes institution-level claims defensible.

Mapping audit expectations to concrete evaluation outputs

The table below maps common institutional-audit expectations — drawn from the ESG and typical institutional-review frameworks — to the evidence reviewers look for and the concrete output that satisfies it.

Audit expectation	What reviewers look for	Concrete Koji output
ESG 1.7 — Information management (collect, analyse, use relevant information)	A systematic, institution-wide dataset that is actually used, not a drawer of surveys	Standardized evaluation data across all modules, automatically analysed into themes and metrics
ESG 1.9 — Ongoing monitoring and periodic review (review programmes; communicate resulting actions)	Periodic review plus documented actions communicated to those concerned	Longitudinal cohort trends + a closing-the-loop action log tied to each finding
ESG 1.3 — Student-centred learning	Evidence the student voice genuinely informs teaching and assessment	Qualitative interview themes on the learning experience, with representative quotes
ESG 2.1 — Effectiveness of internal QA	Proof the internal system works and is used across the institution	Coverage and response-rate dashboards, plus actions-taken tracking, rolled up institution-wide
Institution-level aggregation	One coherent view across programmes, faculties, and years	Cross-programme roll-up reporting from a single standardized dataset
Closing the loop ("you said, we did")	Documented feedback to action to outcome, not just data collection	Action tracking with status, owner, and outcome, exportable as evidence

The through-line is that an audit rewards a managed cycle, not a pile of data. ESG Standard 1.9 is explicit that reviews should lead to improvement and that resulting actions should be communicated to those concerned — collection without a documented response is precisely the gap institutional reviewers probe.

Why longitudinal, standardized data is the hard part

Two failure modes sink institutional audits, and both are about structure rather than effort.

Fragmentation. When feedback lives in a dozen tools — a polling app here, a spreadsheet there, an LMS survey somewhere else — nobody can produce a trustworthy institution-wide picture without weeks of manual reconciliation. The data exists but cannot be aggregated, so the institution ends up asserting its quality culture rather than evidencing it.

Un-analysed qualitative data. The richest evidence of student experience is in open-ended comments, but at institutional scale that text is almost never analysed consistently. A faculty with tens of thousands of comments cannot hand-code them the same way every year, so the qualitative story degrades into a few cherry-picked quotes. Automatic, consistent thematic analysis is what turns that text into longitudinal evidence a reviewer can trust.

Koji addresses both by design: a single standardized evaluation instrument across the institution, AI-moderated interviews that probe why behind each rating, and automatic thematic analysis that applies the same lens to every cohort, every year. Because the moderation is standardized and bias-aware, the resulting dataset aggregates honestly — which is the precondition for any institution-level claim. The same AI interview engine underpins the broader Koji platform used for customer and user research; the education product applies it to the evidentiary demands of higher education.

Building the audit narrative

Evidence is necessary but not self-explaining. A strong institutional submission uses the data to tell a cycle story:

• We asked — standardized evaluation across all programmes, with response rates and coverage shown.
• We learned — themes and trends over several cohorts, at institution and faculty level.
• We acted — specific changes, with owners and dates, tracked in a closing-the-loop log.
• It improved — subsequent cohort data showing the effect of those changes.

Because Koji holds the standardized data, the qualitative themes, and the action log in one place, this narrative can be assembled from the platform rather than reconstructed from memory the term before the site visit. That reconstruction burden — not the data collection itself — is where most audit preparation time is actually lost.

When a competitor or a different approach may fit better

Honesty matters more with this audience than with any other, so a few genuine caveats:

• If your institution already runs a mature, well-staffed institutional-research function with an established survey platform and analysts who hand-build longitudinal reports, you may already produce audit-grade evidence. In that case a new platform is an efficiency and depth play — automating qualitative analysis and closing-the-loop tracking — rather than a gap-filler.
• If your immediate need is a one-off self-evaluation document and you have limited capacity to change systems before the deadline, a specialist QA consultant working with your existing data may be the faster path this cycle, with a platform migration planned afterward.
• If your priority is purely quantitative benchmarking against national datasets (for example a national student survey), a system aligned to that instrument may complement, not replace, a qualitative evaluation platform.

Koji is strongest where you need deep, standardized, longitudinal, cross-programme evidence with automated qualitative analysis and a documented improvement loop. It is not a national benchmarking survey and does not pretend to be one.

Practical steps to institution-ready evidence

1. Standardize the core instrument across programmes so results aggregate honestly, while allowing local questions where needed.
2. Collect both formatively and summatively, so you can show responsiveness within a term and a rigorous end-of-module record.
3. Analyse qualitatively at scale with consistent thematic analysis rather than ad-hoc reading.
4. Log every action taken in response to feedback, with owner, date, and outcome.
5. Report longitudinally, tracking the same measures across cohorts so trends are visible.
6. Assemble the cycle narrative — asked, learned, acted, improved — directly from the platform.

Common pitfalls that weaken audit submissions

Even institutions with strong data undermine their own case in predictable ways. The first is presenting collection as if it were a closed loop: a high response rate proves engagement, not improvement, and reviewers under ESG 1.9 specifically look for the action that followed. The second is inconsistent instruments across faculties, which quietly makes every aggregate figure undefendable. The third is treating qualitative comments as decoration — a handful of quotes — rather than as systematically analysed evidence, which is where the real story of student experience lives. The fourth is leaving evidence assembly to the final weeks before a site visit, when the action trail has already gone cold. A standardized, continuously maintained platform avoids all four, because the loop is documented as it happens rather than reconstructed under deadline pressure.

Related resources

• Student feedback as ESG accreditation evidence
• Course evaluation evidence for programme review and revalidation
• Writing the self-evaluation report: course-evaluation evidence
• Course evaluation evidence for accreditation site visits and peer-review panels
• Does closing the feedback loop actually matter?
• Course evaluation evidence for NVAO accreditation

Ready to build audit-grade, cross-programme evidence from your student feedback? Explore Koji for Education.