Will Students Be Honest With an AI Interviewer? What the Disclosure Research Actually Shows

The short answer: The most common objection to AI-moderated course evaluation is that students will not be candid with a machine. The empirical record points the other way. Across two decades of computer-mediated communication research, people frequently disclose more honestly to a system they believe is automated than to a human, because the fear of being judged falls away. But that advantage is conditional — it depends on genuine anonymity, transparent framing, and a non-judgmental design. Get those wrong and the objection becomes self-fulfilling.

Where the objection comes from

It is a reasonable instinct. Course evaluation is already a low-trust ritual at many institutions: students suspect no one reads the comments, faculty suspect the numbers are noise, and everyone fills in the form to make it disappear. Drop an AI interviewer into that environment and the natural fear is that students will either disengage entirely or perform — telling the machine what they think it wants to hear.

That fear deserves a serious answer rather than a marketing slogan. So let us look at what the disclosure research actually says, then confront the strongest version of the counterargument.

What the disclosure research shows

The cleanest evidence comes from a much-cited experiment by Lucas and colleagues, published in Computers in Human Behavior in 2014. Participants were interviewed by a virtual human and told either that it was controlled by a person or that it was fully automated. Those who believed they were talking to a computer reported lower fear of self-disclosure, lower impression management, and showed objectively greater willingness to disclose (Lucas et al., 2014). The mechanism is intuitive: a computer is not perceived to judge you, so the social cost of an honest, unflattering answer drops.

This is not an isolated finding. It sits on top of a long survey-methodology literature showing that self-administered and computer-administered modes reduce social desirability bias relative to face-to-face or interviewer-administered formats, particularly for sensitive questions (Tourangeau & Yan, 2007, Psychological Bulletin). Reviews of conversational agents reach the same conclusion: assessment by virtual agents, because they afford anonymity, can increase honest reporting on highly sensitive topics such as PTSD symptoms and suicidal ideation. Course feedback is far less sensitive than that — but the same psychology applies to telling the truth about a respected lecturer whose module you found confusing.

Why this matters now

The honesty question only became urgent because the alternative is decaying. Static end-of-term surveys are losing the students they need most. National-scale data show the trend clearly: the National Survey of Student Engagement saw average institutional response rates fall from 42% in 2000 to roughly 25–26% by 2024, and survey requests sent to students have risen by about 71% since 2020 (Hamel & Hennes, 2023, survey-fatigue analysis). When a five-point Likert form takes ninety seconds and feels pointless, the rational student response is to straightline it or skip it.

So the real comparison is not "honest human conversation versus dishonest AI." It is "a perfunctory, half-abandoned tick-box versus a conversation that at least asks a follow-up question." On that comparison, the disclosure literature is encouraging: a non-judgmental interviewer that probes why a student felt lost in week six can surface more truthful, more actionable feedback than a number ever will.

"But doesn't this just assume students trust the AI?"

Here is the strongest counterargument, and it is correct that the picture is not uniformly rosy. The honesty advantage is real but conditional, and the evidence is genuinely mixed in places.

First, some studies find no disclosure difference between human and chatbot conditions. A 2024 study in Interacting with Computers found no difference in the self-reported intimacy of disclosure between talking to a chatbot and a human (Croes et al., 2024). The effect depends heavily on design, framing, and what is being disclosed.

Second, the direction of trust cues matters. Recent work shows that cues suggesting a human is behind the system can either raise or lower disclosure depending on context — there is no guarantee that "more humanlike" means "more truthful." For sensitive questions, a more humanlike agent can actually elicit more socially desirable (less honest) answers, not fewer.

Third, there are two course-evaluation-specific risks the disclosure literature does not address:

• Novelty and distrust. Early enthusiasm can fade, and a minority of students hold a principled distrust of AI handling their words. If they suspect the transcript is not anonymous, the judgment-free advantage evaporates instantly.
• AI-written responses. The same tools that make conversational evaluation possible let students generate plausible-sounding feedback with no real opinion behind it — a data-quality threat we treat in depth in our piece on students using AI to write course feedback.

The honest conclusion is that AI moderation does not guarantee honesty. It removes a specific, well-documented barrier to honesty — fear of human judgment — provided the surrounding conditions hold.

What actually makes the difference

The research converges on a short list of preconditions. Honesty rises when the student believes the channel is anonymous, when the framing is transparent (they know what the AI is and what happens to their words), when the interaction is non-judgmental, and when the conversation is good enough to feel worth their time. Honesty falls when any of those is violated — when anonymity is ambiguous, when the tool feels like surveillance, or when it is so shallow that students conclude, again, that no one is really listening.

This is why the question "will students be honest with an AI?" is really a design question, not a technology question.

How Koji approaches it

Koji for Education is built around exactly those preconditions. The evaluation is an AI-moderated conversational interview, not a free-text box bolted onto a survey — it asks a follow-up when a student says a module was "a bit much," instead of leaving that comment to die in a spreadsheet. Moderation is standardized and bias-aware, so every student meets the same non-judgmental interviewer rather than the variable warmth or impatience of a human one. Data handling is GDPR/AVG-compliant and EU-appropriate, and anonymity guarantees are explicit, because the disclosure advantage only survives if students believe them. And because Koji supports six structured question types alongside open conversation — scale, single- and multiple-choice, ranking, yes/no, and open-ended — institutions keep the comparable metrics they need while gaining the candour the numbers never captured.

The same conversational interview engine powers general customer and user research on the main Koji platform — the education product is that engine, tuned for the realities of higher education.

We are deliberately careful about the claim. Koji does not eliminate dishonesty or social desirability; nothing can. It reduces a specific barrier the evidence has documented for twenty years, and it makes the conditions for candour — anonymity, transparency, non-judgment — the default rather than an afterthought.

Depth matters as much as anonymity

There is a second mechanism in the disclosure research that is easy to miss: it is not only who asks but how they ask. A single Likert item invites a top-of-head answer; a follow-up question invites reflection. When an interviewer — human or AI — probes "you said the pace was fine, but was there a week where you felt lost?", students retrieve and report specifics they would never volunteer on a static form. The conversational-agent literature finds that agents with better conversational ability elicit more considered responses, and that the act of elaboration itself surfaces information a checkbox cannot. So the honesty advantage of AI moderation has two sources, not one: the removal of judgment fear and the depth of a conversation that keeps asking. A non-judgmental interviewer that also probes is a stronger instrument for candour than either property alone.

The bottom line

"Students won't be honest with a machine" is an intuition, not a finding. The findings, taken together, say something more precise and more useful: students are often more honest when they are not afraid of being judged, and a well-designed, transparent, anonymous AI interviewer is a powerful way to remove that fear — as long as institutions earn and keep the trust the design depends on.

If you are weighing whether conversational evaluation belongs in your quality-assurance toolkit, explore Koji for Education — and read our companion analysis on the future of student feedback.