Will Students Open Up to an AI That Runs Their Course Evaluation? Algorithm Aversion, Appreciation, and Disclosure

In brief: Whether people accept advice or judgement from an algorithm is not a simple yes/no. They show algorithm appreciation — often trusting algorithmic judgement over human judgement — until they see the algorithm err, at which point algorithm aversion sets in (Logg, Minson & Moore, 2019; Dietvorst, Simmons & Massey, 2015). For a course evaluation conducted by an AI, the more relevant evidence is about disclosure: people often reveal more, and manage impressions less, when they believe a computer rather than a human is on the other side (Lucas et al., 2014). The design implication is clear: an AI moderator can increase candour, but only if it is transparent, competent, and never positioned as the entity that judges the teacher.

The question this answers

AI-moderated course evaluation — a conversational agent that asks a student follow-up questions, probes a vague answer, and adapts to what they say — promises richer data than a static Likert form. But it raises an immediate objection: will students trust it? Will an AI interviewer make them clam up, game their answers, or dismiss the whole exercise as a gimmick? The behavioural-science literature on how people relate to algorithms is directly relevant, and the answer is more nuanced — and more favourable — than the reflexive scepticism suggests.

What the research says

People often prefer algorithmic judgement — "algorithm appreciation." Logg, Minson and Moore (2019, Organizational Behavior and Human Decision Processes, 151, 90–103) ran six experiments and found that lay people adhered more to identical advice when they believed it came from an algorithm than from a person, across numeric estimates, forecasts of song popularity, and even romantic-match predictions. The received wisdom that people are inherently sceptical of algorithms turned out to be, at minimum, overstated.

But confidence is brittle — "algorithm aversion" after error. Dietvorst, Simmons and Massey (2015, Journal of Experimental Psychology: General, 144(1), 114–126) showed the flip side. When participants saw an algorithm make a mistake, they abandoned it — even when it still outperformed their own judgement, and even when the human alternative erred more. Seeing the model err mattered far more than seeing it succeed. The practical lesson: trust in an algorithm is real but fragile, and visible, unexplained failure destroys it fast.

People sometimes disclose more to a machine. The most directly relevant evidence for AI-conducted evaluation is Lucas, Gratch, King and Morency (2014, Computers in Human Behavior, 37, 94–100). When a virtual-human interviewer was framed as computer-controlled rather than human-operated, participants reported lower fear of self-disclosure, engaged in less impression management, and were rated by observers as disclosing more. Removing the sense of being judged by a person lowered the social-evaluative barrier. For honest course feedback — where fear of identifiability and of hurting a lecturer suppresses candour — this is a genuinely encouraging finding.

Resistance is strongest when people feel their case is unique. Longoni, Bonezzi and Morewedge (2019, Journal of Consumer Research, 46(4), 629–650) found resistance to medical AI is driven by "uniqueness neglect" — a fear that an algorithm cannot account for one's individual circumstances. The lesson transfers: acceptance depends on the AI demonstrably engaging with the individual's specific experience, not delivering generic, one-size-fits-all prompts.

Synthesised: people are not uniformly averse to algorithms; they appreciate competent ones, punish visibly erring ones, disclose more candidly to non-judgemental machine interlocutors, and resist AI that seems to ignore their individuality.

Why it matters for course evaluation in practice

The candour case is real but conditional. The Lucas et al. finding suggests an AI moderator can reduce the social-desirability pressure that distorts human-administered or identifiable feedback (a problem examined in our piece on mode effects and social desirability). Students who fear being recognised, or who soften criticism to be kind to a person, may be more forthcoming with an agent they perceive as non-judgemental. But this depends entirely on the AI being framed honestly and feeling genuinely non-evaluative.

Competence and transparency are non-negotiable. Dietvorst et al. warn that a single visible failure — a follow-up question that misreads the student's answer, a probe that repeats something already said — can trigger aversion and tank engagement for the rest of the interview. An AI moderator has to be genuinely good at understanding responses, and honest about what it is, because recovering trust after a visible error is hard.

"Uniqueness neglect" is a design constraint. Longoni et al. imply that generic, scripted AI prompts will provoke resistance. The value of an AI moderator is precisely that it can engage with the specific thing a student said — which is also what makes it acceptable. A conversational agent that visibly adapts to the individual answer both produces better data and clears the acceptance bar. This is the same reason interactive follow-up probes improve open-text quality.

None of this changes who the AI is accountable to. The AI conducts the conversation; it must never be positioned as the entity that judges the lecturer. Students accept a competent interviewer; they would rightly distrust a system that felt like an algorithm passing verdict on their teacher's career.

Limitations and honest caveats

Generalisability from lab to classroom is unproven. The core studies are lab and applied experiments in forecasting, advice-taking, and health screening — not course evaluation. Transfer is plausible (the social-evaluative and trust mechanisms are general) but not demonstrated for student evaluation specifically. Treat the candour benefit as a well-motivated hypothesis to monitor, not an established fact.

Attitudes to AI are moving fast. All four studies predate the mass adoption of large language models. Students in 2026 have far more direct experience of conversational AI than participants a decade ago, which could push either way — more comfort and fluency, or more savvy gaming and more scepticism about where their words go. The evidence base needs refreshing against current cohorts.

Disclosure has a governance flip side. If students disclose more to an AI moderator, the duty of care around what they disclose grows. Candid feedback may include distress, allegations, or identifiable detail, raising the same anonymity and re-identification questions covered in our k-anonymity guidance — and, in Europe, EU AI Act obligations for AI systems used in education.

Algorithm aversion can still dominate for some students. Appreciation is an average tendency, not a universal one; a subset of students will distrust an AI interviewer regardless, and forcing the modality on them risks non-response bias. Offering a route that does not depend on trusting the agent matters for representativeness.

How Koji incorporates this

Koji for Education is built as an AI-moderated conversational evaluation platform, and these findings map directly onto its design choices.

• Non-judgemental, disclosure-friendly framing. Koji's AI-moderated interview is designed to feel like a neutral listener that helps a student articulate their experience — the conditions Lucas et al. associate with lower impression management and higher disclosure — rather than an authority scoring their teacher.
• Genuine adaptive probing, not scripted prompts. To clear the "uniqueness neglect" bar, Koji probes the specific content of a student's answer (open_ended follow-ups that reference what was actually said), so the interaction demonstrably engages with the individual case rather than delivering generic questions.
• Competence to avoid the aversion trigger. Because Dietvorst et al. show visible error breeds aversion, the interview engine is designed to track context within a conversation to avoid repeating questions or misreading answers — the small failures that would otherwise erode trust mid-interview.
• Transparency about what it is. Koji is designed to be clear that an AI is conducting the conversation, consistent with EU AI Act transparency expectations, because honest framing is what makes the disclosure benefit legitimate rather than a trick.
• Bounded role. The AI moderates the conversation and performs thematic analysis; the judgement about a course or teacher stays with human reviewers reading contextualised, aggregated evidence. Koji is designed to support candid data collection, not to automate personnel verdicts.
• Structured questions alongside conversation. For students who prefer not to converse with an agent, structured items (scale, single_choice, multiple_choice) provide a low-trust-cost route, mitigating the non-response risk from residual algorithm aversion.

The same AI-moderated interview engine powers Koji's core research platform at koji.so, where the identical question — will a respondent open up to an AI interviewer? — is central to product and customer research. We frame the benefit carefully: an AI moderator is designed to reduce social-desirability pressure and elicit richer feedback, not guaranteed to make every student candid.

Related Resources

• Mode Effects and Social Desirability in Conversational Evaluation
• Do Interactive Follow-Up Probes Improve Open-Text Quality?
• Does Anonymity Make Students More Honest?
• Can an LLM Code Your Open-Text Course Feedback?
• When Students Use ChatGPT to Write Their Feedback: Data Integrity
• The EU AI Act and Course-Evaluation Software

References

• Logg, J. M., Minson, J. A., & Moore, D. A. (2019). Algorithm appreciation: People prefer algorithmic to human judgment. Organizational Behavior and Human Decision Processes, 151, 90–103. https://doi.org/10.1016/j.obhdp.2018.12.005
• Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015). Algorithm aversion: People erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General, 144(1), 114–126. https://doi.org/10.1037/xge0000033
• Lucas, G. M., Gratch, J., King, A., & Morency, L.-P. (2014). It's only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior, 37, 94–100. https://doi.org/10.1016/j.chb.2014.04.043
• Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019). Resistance to medical artificial intelligence. Journal of Consumer Research, 46(4), 629–650. https://doi.org/10.1093/jcr/ucz013