Evaluating Engineering Programmes for EUR-ACE: From Satisfaction to Demonstrated Outcomes

Bottom line up front: If your engineering faculty is chasing or holding the EUR-ACE label, your evaluation system has a job most satisfaction surveys cannot do. EUR-ACE accreditation, run under the European Network for Accreditation of Engineering Education (ENAEE), assesses programmes against programme outcomes — what a graduate can demonstrably do — not how happy students were with the course. A five-point "I was satisfied with this module" survey generates almost none of the evidence an accreditation panel wants. This piece maps the EUR-ACE outcome standards, explains why the standard end-of-module survey misfires against them, and sets out what programme-level evaluation should actually capture.

What EUR-ACE actually asks for

ENAEE maintains the EUR-ACE Framework Standards and Guidelines (EAFSG), updated to Edition 4 in November 2021 (ENAEE, EAFSG). The framework is built around three components: student workload requirements (expressed in ECTS), programme outcomes, and programme management. The centre of gravity is the programme outcomes — the knowledge, understanding, skills and abilities an accredited engineering degree must enable a graduate to demonstrate. They are grouped under six headings:

1. Knowledge and understanding
2. Engineering analysis
3. Engineering design
4. Investigations
5. Engineering practice
6. Transferable skills

The standards are also cycle-differentiated. First-cycle (bachelor-level) graduates are expected to apply established methods; second-cycle (master-level) graduates must demonstrate innovative problem-solving and the ability to tackle unfamiliar, incompletely defined problems (ENAEE, EUR-ACE system). The EUR-ACE label carries international recognition and signals that a programme meets these outcome standards.

Read that list again and notice what is not on it: satisfaction, enjoyment, or perceived workload comfort. Every heading is an ability to be demonstrated. That single fact reframes what evaluation is for.

Why the end-of-module satisfaction survey misfires

The typical module survey asks students to agree or disagree that the module was well organised, the lecturer was clear, and they were satisfied overall. None of those items speak to whether a graduate can design an engineering system under constraints, investigate using appropriate experimental methods, or apply engineering practice responsibly. The survey measures the experience of the module; EUR-ACE asks about the outputs of the programme. This is the classic gap between indirect and direct measures of learning — and engineering accreditation leans hard on the direct side.

Two further mismatches make the standard survey particularly weak for engineering:

It is module-shaped, but outcomes are programme-shaped. EUR-ACE outcomes accrue across a whole degree — design capability is built in second year, deepened in the capstone, evidenced in the thesis. A module-by-module survey cannot see that arc, the programme-level blind spot that course-level instruments always have.

It is lecture-shaped, but engineering is lab-, studio- and project-shaped. A form written for a lecture theatre fits the design studio, the teaching lab and the group project badly — a pedagogy mismatch we describe in evaluating lab, studio and clinical teaching. Engineering practice and investigation outcomes are forged precisely in those settings the generic survey handles worst.

What programme-level evaluation should collect

EUR-ACE's third component — programme management — expects a functioning continuous-improvement loop that includes student and stakeholder feedback. That is where evaluation earns its place, provided it is redesigned around outcomes rather than satisfaction. Practically:

• Ask about demonstrated capability, not comfort. Alongside direct assessment, collect students' evidence-referenced perceptions of the six outcome domains: where in the programme did you actually practise engineering design? Can you point to it? This mirrors the assurance-of-learning discipline that business-school accreditation has used for years — engineering simply has a different, EUR-ACE-shaped outcome map.

• Map items to outcomes explicitly. Every evaluation question should trace to a programme outcome and, through constructive alignment, to the assessments that generate direct evidence. An unmapped question is noise on an accreditation dashboard.

• Capture transferable skills where exams are blind. Communication, teamwork, ethics and lifelong-learning — the transferable-skills heading — are exactly the outcomes written exams measure poorly. Well-designed student and employer perception data is legitimate indirect evidence here, and connecting it to a skills taxonomy such as ESCO makes it auditable.

• Move beyond satisfaction as the headline. The whole point is captured in the shift from measuring satisfaction to measuring skill development.

But doesn't accreditation want direct evidence, making student feedback irrelevant?

This is the strongest objection and it is half right. EUR-ACE panels do prioritise direct evidence of outcomes: examination scripts, design-project reports, thesis assessment, capstone rubrics. No student survey substitutes for a marked design portfolio, and it should not try to.

But three things keep indirect evaluation firmly in scope. First, EUR-ACE's programme-management component explicitly expects a continuous-improvement cycle fed by student and stakeholder input — the loop is part of the standard, not an optional extra, and it sits inside the wider ENQA/ESG expectation that European programmes act on student feedback. Second, direct measures answer whether an outcome was achieved but are often silent on why a cohort underperformed; indirect evaluation is where the diagnostic signal lives. Third, for transferable skills and engineering practice, well-constructed perception evidence is among the more valid indirect measures available, because those outcomes are enacted in projects and placements rather than in exams.

So the honest framing is not "direct versus indirect". It is: direct measures are the primary evidence of attainment; indirect evaluation is the primary evidence of why, and the required mechanism for improvement. A programme that produces only direct evidence can prove its graduates are competent this year but cannot show the reflective, closing-the-loop management EUR-ACE also demands.

Where Koji fits

Turning a satisfaction survey into outcome-referenced, accreditation-ready evidence is exactly the problem Koji for Education is built for. Instead of a Likert grid, Koji runs AI-moderated conversational interviews that can be structured around the six EUR-ACE outcome headings and then probe for evidence: not "were you satisfied with the design module?" but "describe a point in the programme where you designed something under real constraints — what did you have to weigh?" Across six structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no), you capture both comparable ratings and the concrete, quotable evidence a panel finds credible.

Its automatic thematic analysis then clusters those responses by outcome domain, quality-scored and with the underlying quotes attached, so a self-assessment report can state — and show — where in the programme each outcome is practised and where students report gaps. Because reporting rolls up from module to programme and institution level, evaluation finally matches the programme-shaped grain EUR-ACE assesses on, and because collection can be formative and mid-cycle, the continuous-improvement loop is real rather than retrospective. The moderation is standardised, bias-aware and GDPR-compliant with EU-appropriate data handling — details a European accreditation context takes seriously.

Faculties that also run graduate-outcome or employer research beyond accreditation will find the main Koji platform uses the same AI interview engine, so the same evidence discipline extends to alumni and industry feedback.

EUR-ACE does not ask whether your students were satisfied. It asks whether your graduates can demonstrably do six things. Build an evaluation system that answers that question — and can show its working.

Mapping the six outcomes to evaluation questions

The practical test of an outcome-referenced system is whether every question earns its place against a EUR-ACE heading. A workable mapping looks like this: for engineering design, ask students to point to where in the programme they designed under real constraints and what they had to trade off; for investigations, where they planned and ran experiments and interpreted ambiguous data; for engineering practice, where they used professional tools, codes and standards; for transferable skills, where they led, wrote or presented to a non-specialist audience. Satisfaction never appears; evidence of enacted capability does. The workload component matters too: EUR-ACE expresses study requirements in ECTS, and course evaluation is one of the few instruments that can surface when the credit does not match the clock — an ECTS workload validity check that accreditation panels increasingly probe. An evaluation form built this way produces, as a by-product, a defensible map of where each outcome is practised across the degree — precisely the artefact a self-assessment report needs and a generic satisfaction survey can never assemble.

Preparing an engineering programme for EUR-ACE review? See how Koji turns feedback into outcome evidence.