You Cannot Survey Your Way to an Architect: Evaluating Design Education Under the ARB's New Outcomes

Bottom line: Architecture is a regulated profession whose teaching looks nothing like a taught lecture course — it happens in the design studio and the crit. The generic end-of-module satisfaction survey misreads it twice over. It cannot evidence the regulated competences the qualification actually certifies, and it actively penalises the studio's signature pedagogy, in which productive discomfort is the method rather than a defect. And the ground is moving: the UK's Architects Registration Board (ARB) is replacing the familiar Part 1 / Part 2 / Part 3 structure with a set of competency outcomes, mandatory for new entrants from September 2027. An evaluation system that can only report whether students were satisfied will have nothing to say to an outcomes-based regulator.

What the qualification actually certifies

Architecture is one of the professions with automatic recognition across the EU under Directive 2005/36/EC. Article 46 sets the shape of the training: at least five years of full-time study, or four years full-time plus a two-year traineeship, with architecture as the principal component. Crucially, Article 46 does not describe hours of satisfaction — it lists the abilities and areas of knowledge a graduate must acquire: the ability to create architectural designs that satisfy both aesthetic and technical requirements; adequate knowledge of the history and theories of architecture; an understanding of the relationship between people, buildings and their environment; and knowledge of structural design, construction and engineering problems, among others. The qualification is a certification of competence, in the same way a regulated-profession degree is a passport. None of those competences is visible in "How satisfied were you with this module?"

The ARB reform makes this urgent

For decades UK architectural education has been structured as Parts 1, 2 and 3. The ARB is dismantling that. Following its statutory consultation, the ARB approved a new competency-based framework built around a set of competency outcomes demonstrated across two post-degree stages — an academic/learning stage and a practice stage. The regulatory focus shifts explicitly to what an individual must know, what they must be able to do, and how they must behave, rather than how or what they were taught. New qualifications must deliver these competency outcomes from January 2024, and from September 2027 anyone starting the route to registration must do so under the new system.

That is a decisive move from inputs to outcomes, and it changes the evidence a programme has to produce. An outcomes regulator does not ask whether students enjoyed the studio; it asks whether the programme can show its graduates meet defined competences. A satisfaction mean is silent on that question. The institutions that will struggle in 2027 are the ones whose entire evaluation apparatus was built to produce a number the new framework does not ask for.

The studio penalises the very instrument you are using

Even setting the regulator aside, the survey mismeasures the pedagogy. The design studio and the crit are the signature pedagogy of architecture — the format, going back to Schön's account of the reflective practitioner, in which students learn by making, presenting, and defending work under critical questioning. The crit is supposed to be uncomfortable; being challenged on your design decisions in front of peers is how architectural judgement is formed. A satisfaction instrument systematically misreads this: it reads rigour as hostility, rewards the tutor who went easy, and penalises the one whose demanding crit produced the most growth. This is the same trap that catches lab, studio and clinical teaching and practice-based arts education — but architecture stacks a statutory competence framework on top, so the cost of mismeasurement is regulatory as well as pedagogical.

What good evaluation asks instead

The useful questions are not about satisfaction but about development and fairness. Did the crit feedback help the student improve their design, or was it demoralising without being constructive? Those are different things, and only one is a problem. Is technical knowledge genuinely integrated with design, as Article 46 requires, or bolted on? Is the studio culture inclusive — and is it survivable? Architecture education has a well-documented culture of overwork and studio-related wellbeing strain, and here student voice matters enormously; that is an argument for listening better, not for a happiness score. Answering these questions means qualitative evidence, gathered formatively while a studio is still running, mapped to the competences the programme must evidence.

The counterargument, taken seriously

Two objections deserve a straight answer. First: satisfaction data is not worthless. A studio that scores catastrophically is telling you something real — often about workload, wellbeing, or a genuinely broken tutor relationship — and dismissing all student sentiment would be its own failure, especially given architecture's wellbeing record. Fair. The claim is not that student voice is irrelevant; it is the opposite. It is that a single satisfaction number is the wrong instrument for a discipline whose feedback is developmental and whose pedagogy is deliberately demanding — you need richer signal, not less of it.

Second: outcomes frameworks can be reductive. Ticking a graduate against 49 competences risks flattening design judgement into a checklist, and no one becomes an architect by accumulating boxes. This is a real danger, the same one that shadows outcomes-based engineering accreditation. But the answer is not to cling to a satisfaction survey that evidences neither judgement nor competence; it is to gather qualitative, defensible evidence of how competence is developing and map it to the outcomes, so the regulator sees substance rather than a spreadsheet.

Where Koji fits

The distinctions that matter in a studio — tough versus unfair, demanding versus demoralising, challenging-and-constructive versus challenging-and-pointless — are exactly the ones a Likert scale collapses and a conversation preserves. Koji's AI-moderated conversational interviews probe them: when a student says the crit was "brutal," the interview asks whether it helped, surfacing the difference the number hides. Automatic thematic analysis turns open-text accounts of studio culture and workload into patterns a programme director can act on, and mapping feedback to competency outcomes builds precisely the evidence base the ARB's new framework will demand. Formative, mid-studio timing catches a failing studio while there is time to intervene; programme-level reporting shows how a whole cohort is developing against the outcomes rather than how happy they were this week. And because the same interview engine powers wider stakeholder and practice research on the main Koji platform, schools that also gather alumni or employer input will find one consistent method across all of it.

You cannot survey your way to an architect — and after 2027, you will not be able to satisfaction-score your way past the regulator either. The programmes that thrive will be the ones already listening in a way that evidences competence, not just contentment. That is a shift in what evaluation is for, and the studio is where it starts.

Evaluating a design programme for the ARB's new outcomes? Koji for Education captures the developmental, competence-level evidence a satisfaction survey never could.

Frequently asked questions

What does EU law require of an architecture programme? Directive 2005/36/EC, Article 46, requires at least five years of full-time study (or four years plus a two-year traineeship) with architecture as the principal component, and specifies the abilities and knowledge a graduate must acquire — design satisfying aesthetic and technical requirements, history and theory, structural and construction knowledge, and more.

What is changing with the ARB's reforms? The ARB is replacing the Part 1 / Part 2 / Part 3 structure with a competency-outcomes framework demonstrated across two stages. New qualifications must deliver the competency outcomes from January 2024, and from September 2027 all new entrants to the route must qualify under the new system.

Why does a satisfaction survey misread the design studio? The studio and the crit are a signature pedagogy in which challenge is the method. A satisfaction instrument reads rigour as hostility, rewards easy tutors, and penalises demanding ones — so it can invert the relationship between good teaching and high scores.

Isn't student satisfaction still worth measuring, given wellbeing concerns? Student voice matters enormously in architecture, precisely because of its overwork culture — but that is an argument for richer, more diagnostic listening, not a single happiness score. You want to distinguish a demanding studio from a harmful one, which a mean cannot do.

How should architecture programmes evidence competence for the regulator? With qualitative, developmental evidence gathered formatively and mapped to the required competency outcomes — showing how students' design judgement and technical integration develop over time, rather than reporting an end-of-module satisfaction mean the outcomes framework does not ask for.