Closing the Loop Starts With the Teacher: Feedback Literacy for Reading Course Evaluations

The short answer: Most course-evaluation reform focuses on collection — response rates, item wording, timing — and on institutional process — action plans, quality cycles. It rarely addresses the person who actually has to read the results and change their teaching: the academic. The concept of teacher feedback literacy, developed by David Carless and Naomi Winstone, describes the knowledge, capabilities and dispositions needed to make productive use of feedback. Applied to course evaluation, it reframes the 'feedback loop' problem: a loop cannot close if the person at the receiving end lacks the literacy to interpret noisy, emotionally charged, multilevel data and translate it into pedagogical change. Building that literacy — and giving academics feedback that is interpretable in the first place — is the highest-leverage and most neglected move in the whole system.

We built the pipes and forgot the reader

Walk through a typical university's course-evaluation apparatus and you will find enormous effort spent on the front of the pipe. Committees agonise over the questionnaire. Response rates are chased with reminders, incentives and, sometimes, mild coercion. Dashboards are commissioned. Quality-assurance frameworks specify that results must be reviewed and 'actioned'. Then, at the far end of all this machinery, a document lands in an academic's inbox: a page of means and distributions and a scroll of anonymous free-text comments, some warm, some wounding, most ambiguous. And the institution assumes — without training, without support, often without even a conversation — that this person knows how to read it well and will change their practice accordingly.

That assumption is the quiet failure point of 'closing the loop'. The loop does not close in the dashboard. It closes, or fails to, in the head of the person reading the report. And reading evaluation feedback well is a genuine skill that almost no one is taught.

Borrowing a concept that fits exactly

The assessment literature has, over the past decade, developed precisely the concept we need. Building on their work on student feedback literacy, Carless and Winstone (2020/2023) proposed a framework for teacher feedback literacy, defining it as the 'knowledge, expertise and dispositions to design feedback processes' that enable uptake. Their framework has three dimensions: a design dimension (structuring feedback so it can actually be used), a relational dimension (the interpersonal, emotional side of feedback exchange), and a pragmatic dimension (managing the workload and institutional constraints around feedback).

Carless and Winstone were writing about teachers giving feedback to students. But the framework transfers almost intact to teachers receiving feedback from students through course evaluation — and the transfer is illuminating, because each of their three dimensions maps onto a specific way that evaluation loops break.

The relational dimension: feedback is emotional, and defensiveness is the default

Carless and Winstone put the relational dimension at the centre for good reason: feedback is an emotional transaction, not a data transfer. A page of student comments can wound, and the reflexive response to a stinging comment is defence — dismiss the student as disgruntled, attribute the criticism to a hard grade, discount the whole report. Feedback-literate academics recognise this reaction in themselves and have strategies to metabolise it: separating the sting of a comment from its potential informational value, resisting the pull of the single cruel remark (the well-documented negativity bias in reading open text means one harsh comment outweighs ten kind ones in memory), and returning to the data once the emotional charge has cooled. Institutions that dump raw, unmediated comments on staff with no relational support all but guarantee defensiveness rather than uptake.

The design dimension: uninterpretable feedback cannot be acted on

The design dimension is where institutions have the most leverage. Much evaluation feedback is simply not designed for uptake. A bare mean of 3.8 answers no actionable question. Which aspect of the course? For whom? Compared to what reasonable benchmark? Is 3.8 different from last year in any meaningful, uncertainty-adjusted sense, or is it noise — the kind of regression-to-the-mean movement that means nothing? Feedback-literate systems present results so that an academic can locate a specific, changeable practice: themed open-text analysis pointing to a recurring issue, sub-group patterns, and change flagged with appropriate uncertainty rather than false precision. You cannot expect literate reading of illiterate reporting.

The pragmatic dimension: acting on feedback competes with everything else

The pragmatic dimension names the obvious constraint: academics are overloaded, and reflecting deeply on evaluation data competes with research, admin and the next term's preparation. Feedback literacy here means triage — identifying the one or two highest-value changes a report actually supports, rather than either ignoring the whole thing or being paralysed by trying to respond to every comment. Institutions support this by making the signal easy to find, not by adding another form to complete.

The interplay with student feedback literacy

Carless and Winstone stress that teacher and student feedback literacy are interdependent, and the same holds in evaluation. Students who understand what constructive, specific feedback looks like write more usable comments; academics who demonstrably act on feedback earn the credibility that motivates students to bother responding at all. This is the antidote to the widely observed problem that students are willing to evaluate but doubt their feedback is used. The loop is a relationship, not a data pipeline — and relationships require literacy on both ends.

But isn't this just blaming academics for not reading their reports?

The sharpest objection is that 'build teacher feedback literacy' can become a way of offloading a structural problem onto individuals — telling overworked staff that if only they read their evaluations better, the system would work. That would be a misreading. Feedback literacy is not solely a property of the individual; Carless and Winstone's design and pragmatic dimensions are explicitly institutional responsibilities. If the report is an uninterpretable wall of numbers, delivered with no support, at the busiest point of term, the failure is systemic, not personal. The argument is not 'train academics to cope with bad feedback'. It is 'recognise that the loop closes in a human reader, and design the whole system — the data, the delivery, the support — around making that reader's interpretation successful.' Literacy is co-produced by the person and the environment.

A second objection: doesn't rich, conversational feedback make the reading harder, not easier — more to process than a tidy number? Only if it arrives unprocessed. The point is to pair depth of evidence with analysis that renders it interpretable, so the academic gets both the 'what' and a route to the 'so what'.

How Koji supports the reader, not just the survey

Koji for Education is built on the premise that the report is where evaluation succeeds or fails. Rather than handing academics a raw comment dump, Koji's automatic thematic analysis organises open-text feedback into recurring themes with representative quotes, so the reader sees the pattern rather than being ambushed by the single cruel line — directly supporting the relational and design dimensions of feedback literacy. Its AI-moderated conversational interviews produce feedback that is specific by construction: because the AI probes 'can you give an example?', comments arrive attached to concrete, changeable practices rather than vague verdicts, which is exactly the design property that makes feedback actionable. Quality scoring and thematic prioritisation help with the pragmatic dimension — surfacing the one or two highest-value signals instead of an undifferentiated list. And because the same standardized moderation is applied to every student, academics can trust that the feedback reflects the course rather than inconsistent survey administration.

Koji cannot make the emotional part of receiving criticism disappear — nothing can, and the relational work of metabolising feedback remains human. What it can do is ensure the feedback that lands is interpretable, themed, specific and prioritised: the conditions under which feedback literacy can actually operate. The same interview-and-synthesis engine powers Koji's main research platform, where the identical truth holds — insight only counts when the person receiving it can act on it.

The takeaway

Response rates and dashboards are the easy, visible part of course evaluation. The hard part is the reader: the academic who must turn noisy, emotional, multilevel feedback into better teaching. Teacher feedback literacy — jointly built by the individual and the institution — is the concept that names this gap. Close it, and evaluation becomes a learning process. Ignore it, and you will keep collecting feedback that no one is equipped to use.