Evaluating Practice-Based Arts Education: The Crit, the Studio & Why SET Fails

Answer first: The standard course-evaluation instrument — a battery of Likert items about "the quality of teaching" — is built around a specific, unstated model of a course: someone delivers content, students absorb it, an exam checks it. Practice-based arts education does not work like that. Its core pedagogy is the critique ("the crit"), the design studio, the one-to-one instrumental lesson, the atelier. Feedback there is continuous, dialogic, embodied, and often deliberately uncomfortable. A satisfaction survey administered at the end of the semester can neither see this teaching nor evaluate it — and worse, because it rewards comfort, it can systematically penalise exactly the rigorous, standards-raising feedback that defines good studio teaching. If you run an art school, a conservatoire, or a design faculty and evaluate it with the same form as the law lecture down the road, you are not measuring quality. You are measuring how much your generic instrument mismatches your pedagogy.

The studio is a signature pedagogy, not a delivery method

Lee Shulman's influential idea of signature pedagogies — the characteristic forms of teaching that initiate students into the thinking of a profession — has been applied directly to art and design, where scholars identify the studio as the signature pedagogy and the crit as its central assessment-and-feedback ritual (Journal of Learning Design; Motley, Arts and Humanities in Higher Education). In this literature the critique is defined not as a lecturer's verdict but as a "structured, student-focused learning activity that serves as an assessment and generator of critical feedback," clarifying the discipline's values and showing students how professionals reach judgements.

Two features of this pedagogy break the standard evaluation instrument:

Feedback is the curriculum, not an add-on. Work on "signature feedback practices in the creative arts" argues that feedback is integrated within the curriculum rather than bolted on at the end (Assessment & Evaluation in Higher Education). There is no clean separation between "teaching" and "assessment" to rate independently — the crit is simultaneously both.
The relationship is often one-to-one and long-running. In a conservatoire, a student may study with the same instrumental teacher for years. The unit of evaluation is not "a module" but a sustained pedagogical relationship — which a module-level, once-a-semester Likert form is structurally incapable of capturing.

Europe's higher music education sector has organised around exactly this specificity: the Association Européenne des Conservatoires (AEC), the sector body for higher music education across the continent, has spent years developing learning-outcomes and quality-enhancement frameworks because generic higher-education quality tools fit conservatoire teaching so poorly.

How the generic survey actively misfires

This is not merely a case of "the survey doesn't capture everything." The mismatch produces biased signal:

It rewards comfort over challenge. A crit that told a talented student their portfolio was not yet good enough — and was right — may score low on satisfaction. A gentle, undemanding tutor may score high. If the programme is managed to those numbers, it selects against rigour. This is the leniency/likeability problem, amplified by a pedagogy whose whole point is honest, standards-referenced critique.
It flattens a dialogue into a rating. Asking a student to compress a semester of studio conversation into "I was satisfied: agree/disagree" discards precisely the qualitative texture — what shifted in their practice, which feedback landed, where the culture supported or blocked risk-taking — that is the actual evidence of teaching quality.
It ignores studio culture. Much of the learning in practice-based education happens horizontally, between peers, in the shared physical space. A teacher-centred rating form cannot see the studio as a learning environment at all.

We have made adjacent arguments before — that teaching portfolios are not the objective alternative people assume, and that service-quality models like SERVQUAL and HEdPERF import assumptions that do not always transfer. The arts case is the sharpest version: the instrument's implicit model of "a course" is simply false for this pedagogy.

What to evaluate instead

The answer is not "arts teaching is unmeasurable, so stop evaluating." It is to evaluate the constructs that practice-based pedagogy actually turns on:

Feedback quality and feedback literacy — did the crit help the student understand the discipline's standards and act on them? This aligns arts evaluation with the broader move toward feedback that students can use, not just receive.
The learning relationship — for one-to-one tuition, the quality, consistency, and developmental arc of the teaching relationship over time, not a module snapshot.
Studio culture and psychological safety — can students take creative risks, show unfinished work, and disagree, without fear? This is an environmental construct, best surfaced through open, narrative response.
Development of practice — students' own accounts of how their work changed, which is qualitative and longitudinal by nature.

Nearly all of this is narrative evidence, gathered formatively, at the level of the relationship or the studio rather than the module — a completely different data shape from a five-point scale.

The strongest counterargument

"Arts programmes still sit inside a university that needs comparable, accountable metrics. You cannot opt out of the institutional dashboard." This is real, and pretending arts education floats free of institutional accountability helps no one. Deans need to compare, quality offices need evidence for accreditation, and "our pedagogy is too special to measure" is exactly the kind of exceptionalism that leaves arts faculties without a defensible evidence base when budgets are cut. The honest resolution is not to abandon evaluation but to change what counts as evidence: structured qualitative data, systematically analysed, is more accountable — not less — than a satisfaction average that everyone privately knows is meaningless for a studio. A well-analysed corpus of student accounts of their crit experience, with themes and prevalence, stands up to scrutiny far better than "4.1 out of 5." The goal is rigorous evaluation that fits the pedagogy, not an exemption from evaluation.

A second objection: "Isn't narrative feedback about a one-to-one teacher hopelessly awkward and biased — the student knows exactly who will read it?" Yes, and that is a genuine risk unique to small, relational settings; it overlaps with the small-class identifiability problem. It is an argument for careful, confidentiality-aware, standardised collection — not for defaulting back to a number that is also biased and additionally uninformative.

Where Koji fits

Practice-based evaluation is a qualitative problem at scale, which is precisely what Koji for Education is built for. Its AI-moderated conversational interviews are far better suited than a Likert grid to eliciting a student's account of a crit, a studio culture, or a long instrumental relationship — the AI probes ("what specifically changed in your work after that feedback?") in a way a static form never can, and it does so consistently across every student, removing the human-moderator inconsistency and awkwardness of a tutor gathering feedback on their own one-to-one teaching. Automatic thematic analysis turns hundreds of these narratives into structured themes with traceable provenance, giving arts faculties the accountable, comparable evidence base the institution demands without collapsing everything into an average. Its six structured question types and formative, mid-cycle collection let you evaluate the relationship as it develops rather than autopsy it at the end.

Koji does not pretend to measure creativity, and it does not replace the professional judgement at the heart of a crit — it surfaces and structures the student's side of that pedagogy so programmes can act on it. It reduces the mismatch between instrument and pedagogy; it does not eliminate the genuine difficulty of evaluating an art. (The same conversational engine runs on the main koji.so platform for any team doing open-ended qualitative research — the education product simply tunes it to the studio.)

The one-paragraph version

Your evaluation form assumes a lecture-and-exam course; your art school, conservatoire, or design studio is not one. The crit and the studio are a signature pedagogy in which feedback is the curriculum, and a satisfaction survey both misses it and penalises the rigour that defines it. Evaluate the real constructs — feedback quality, the learning relationship, studio culture, development of practice — with structured, formative, narrative data. That is more accountable than a meaningless average, not less.

Frequently asked questions

Why does a standard course-evaluation survey fail for arts and design programmes? Standard surveys assume a lecture-and-exam model. Practice-based arts education runs on the studio and the crit, where feedback is continuous, dialogic, and integrated into the curriculum. A once-a-semester Likert form cannot see this teaching and, by rewarding comfort, penalises the demanding feedback that defines it.

What is a signature pedagogy in this context? Lee Shulman's concept describes the characteristic teaching forms that initiate students into a profession's thinking. In art and design, scholars identify the studio as the signature pedagogy and the critique (crit) as its central assessment-and-feedback ritual — a structured, student-focused activity, not a lecturer's verdict.

How can a satisfaction survey actively harm arts teaching? It rewards comfort over challenge. A crit that honestly tells a student their work is not yet good enough may score low, while a gentle, undemanding tutor scores high. Managing to those numbers selects against the rigour good studio teaching requires.

What should arts programmes evaluate instead? Feedback quality and feedback literacy, the developmental arc of the one-to-one learning relationship, studio culture and psychological safety for creative risk-taking, and students' own accounts of how their practice developed — as structured, formative, narrative evidence.

Isn't narrative feedback less accountable than a comparable score? No — a well-analysed corpus of student accounts, with identified themes and their prevalence, stands up to accreditation scrutiny better than a satisfaction average everyone privately knows is meaningless for a studio.

How does an AI-moderated approach help with one-to-one teaching feedback? Conversational interviews probe a student's account of a crit or long instrumental relationship far more richly than a static form, and do so consistently across every student — removing the awkwardness of a tutor collecting feedback on their own teaching. Confidentiality-aware collection also mitigates the identifiability risk in small settings.

Run an arts, design, or music programme and tired of a survey that does not fit your pedagogy? See how Koji for Education evaluates practice-based teaching.