Vague Comments Are a Design Problem: Student Feedback Literacy and the Quality of Course Evaluations

The most common complaint about open-ended course-evaluation comments is that they are useless — too vague to act on ("it was fine"), too emotional to trust ("worst module ever"), or too sparse to matter. The usual conclusion is that students cannot, or will not, give good feedback. That conclusion is wrong, and the mistake is consequential. Giving useful feedback is a learned capability — what the assessment literature calls feedback literacy — and a blank comment box at the end of term scaffolds it not at all. Unhelpful evaluation comments are largely a design problem, not a student deficit. Fix the design and the quality of what students tell you changes.

Feedback literacy is a real, researched construct

The idea comes from a well-developed strand of higher-education research, most influentially David Carless and David Boud's 2018 framework in Assessment & Evaluation in Higher Education. They define student feedback literacy as "the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies," and propose four inter-related features: appreciating feedback (understanding its purpose and one's active role), making judgements (developing the capacity to evaluate quality), managing affect (handling the emotions feedback provokes), and taking action (Carless & Boud, 2018). Crucially, they observe that "inadequate levels of student feedback literacy impede students' productive integration of feedback" — and the same logic runs in reverse when students are the ones giving feedback about a course.

Think about what a course evaluation actually asks of a student. To write a useful comment, a student must: recognise that their feedback has a purpose and an audience (appreciating); form a considered judgement about what worked and why, rather than an affective reaction (making judgements); separate "I found this hard and frustrating" from "this was badly taught" (managing affect — these are different claims); and frame the comment as something actionable (taking action). That is a sophisticated set of capabilities. We spend enormous effort teaching students to receive feedback well. We spend almost none teaching them to give it — and then we are surprised when a one-line text box yields one-line answers.

Why the blank box fails

The end-of-term evaluation is almost perfectly designed to suppress feedback literacy. It arrives after the course is over, when nothing the student says can help them. It offers no prompt richer than "Any other comments?", which signals that anything will do. It gives no indication of what a useful answer looks like, so students default to a verdict ("good," "boring") rather than evidence. And it provides no dialogue — no way for a vague comment to be gently pushed toward specificity. A student who writes "the feedback was unhelpful" is never asked the one question that would make the comment actionable: unhelpful how — too late, too generic, too harsh?

This is why so much open-text data is thin. It is not that students have nothing to say; it is that the instrument asks for a verdict and provides no scaffold for anything better. The related failure mode is at the other extreme: with no structure and the cover of anonymity, a minority of comments tip into the abusive and non-constructive (a problem we examine in abusive and non-constructive evaluation comments). Both the empty and the abusive comment are symptoms of the same missing scaffold.

Designing for feedback literacy

If giving good feedback is a capability, the evaluation should be designed to elicit and support it. The principles map directly onto the Carless–Boud framework:

• Signal purpose and audience (appreciating). Tell students who reads this, what changed last time because of student feedback, and that their specific experience — not a score — is what is useful. Closing the loop visibly is itself a feedback-literacy intervention: students give better feedback when they believe it is used.
• Prompt for evidence, not verdicts (making judgements). Replace "Any other comments?" with specific, behaviour-anchored prompts: "Describe a moment in this course when you felt you learned something difficult. What made it work?" Specific prompts produce specific, usable answers (see open-ended question design).
• Separate experience from evaluation (managing affect). Let students name a frustration and reflect on its cause, so "I was stressed" can be distinguished from "the assessment was poorly structured." These are different data and should be captured as such.
• Make it a dialogue, not a form (taking action). The single biggest lever on comment quality is the follow-up question. A vague answer is not a dead end if something can ask "can you say more about that?"

That last principle is the one a paper form or a static online survey structurally cannot deliver. A form cannot probe. And probing is exactly what turns "the feedback was unhelpful" into "I got my mark three weeks late with no comments, so I couldn't use it for the next assignment" — a sentence a programme team can actually act on.

But isn't this just making excuses for lazy students?

The obvious objection: plenty of students can articulate clearly when they want to, so isn't "feedback literacy" a polite way of excusing low effort? And won't scaffolding and prompting just lead students toward saying what we want to hear?

Two honest responses. First, effort and capability are not the same thing, and even capable students respond to incentives and design. A student who can write beautifully in a graded essay will still write "fine" in a box that arrives after the course, promises nothing, and asks for nothing specific. That is rational, not lazy — the design has told them their answer does not matter. Second, the leading-questions risk is real and must be managed carefully: probing for specificity is legitimate ("can you give an example?"), but probing for a valence ("so it was bad, then?") is not. The distinction is the difference between good interviewing and a leading one — a line we take seriously (see AI interviewer sycophancy and leading questions). Well-designed probing asks the student to elaborate their own judgement; it does not supply one.

A fair limitation: scaffolding raises the floor on feedback quality, but it cannot manufacture insight where a student genuinely has none, and it adds a few seconds of effort that some students will decline. The aim is not to make every comment brilliant — it is to stop wasting the genuine signal that the blank box currently throws away.

Where Koji fits

A blank box cannot scaffold feedback literacy; a conversation can. This is the core of what Koji for Education does differently. Its AI-moderated conversational interviews replace the dead text box with a dialogue: when a student gives a vague answer, the AI asks a neutral, specificity-seeking follow-up — "you mentioned the feedback wasn't useful; what would have made it useful?" — exactly the move that converts a verdict into actionable evidence, applied consistently across an entire cohort with standardized, bias-aware moderation rather than a leading human interviewer. The six structured question types let you design behaviour-anchored prompts rather than catch-all boxes, automatic thematic analysis distils thousands of now-richer responses into the recurring themes a committee can act on, and quality scoring flags low-information responses so you know how much signal you actually have. The same conversational engine powers the main koji.so platform, where the identical principle — probe for the why, never the verdict — drives better customer and user research. Done well, this does more than improve your data: a student asked a good question, and asked to elaborate their own reasoning, is practising feedback literacy itself.

The bottom line

Stop blaming students for the comments your instrument was designed to produce. Useful feedback is a learned capability that a blank, after-the-fact text box actively suppresses. Signal that feedback is used, prompt for evidence rather than verdicts, separate experience from evaluation, and — above all — make it a dialogue that can probe. Get the design right and you will discover students had a great deal to tell you. The box was the problem, not them.