Anonymous Until Someone Reads It: How Writing Style Can De-Anonymise "Anonymous" Course Feedback

Bottom line up front: Removing student names from open-text course feedback does not make it anonymous. Writing style is itself an identifier. In a small, closed cohort — the fifteen to forty students enrolled in a module — distinctive phrasing, idiosyncratic vocabulary, recurring errors, first-language transfer patterns, and references to specific shared moments can make a comment''s author inferable, sometimes by the one reader with the strongest priors of all: the lecturer being evaluated. Under the GDPR''s "means reasonably likely to be used" test, stylometrically identifiable feedback may still be personal data. And the mere belief that one can be recognised chills the honest criticism the whole exercise exists to collect.

The comfortable assumption, and why it is wrong

Almost every course-evaluation policy rests on a promise: your feedback is anonymous. Operationally, "anonymous" nearly always means "we did not attach your name." Ratings are pooled; free-text comments are detached from the roster; a threshold suppresses reports for very small classes. Institutions then treat the result as genuinely anonymous data — outside data-protection obligations, safe to hand to the instructor verbatim.

The assumption is that identity lives in the name field. It does not. Identity also lives in how people write. The study of inferring authorship from linguistic style — stylometry — is old enough to have unmasked disputed Federalist Papers and modern enough to worry security researchers. In 2012, Arvind Narayanan and colleagues published "On the Feasibility of Internet-Scale Author Identification" at the IEEE Symposium on Security and Privacy. Analysing writing style alone, their classifiers correctly identified an anonymous author in over 20% of cases drawn from a pool of 100,000 candidate authors, and placed the true author in the top 20 guesses about 35% of the time. Their explicit warning: an anonymous blogger or whistleblower can be unmasked by style unless they actively obfuscate how they write.

Course feedback is short, but the threat model is far easier than 100,000 strangers.

Why the "you need huge datasets" objection cuts the wrong way

The immediate, sophisticated objection is that internet-scale stylometry needs long documents and struggles on short text; a 150-word course comment is thin, and academic accuracy drops as texts shorten and candidate pools open. All true — for the hard version of the problem Narayanan studied: many words, tens of thousands of unknown candidates, no side information.

But that is not the situation in a course evaluation, and the differences all make identification easier, not harder:

• A tiny, closed candidate set. The author is not one of 100,000 strangers. They are one of the 25 students on the register. Even weak stylistic signal narrows a set that small dramatically; you are not searching a haystack, you are choosing among a handful of people you already know.
• A reader with maximal priors. The person most likely to read the comments is the lecturer, who has spent a term reading these students'' essays, hearing them speak, and learning their preoccupations. Machine stylometry approximates what this reader does effortlessly: "that phrasing, that hobby-horse, that particular grammar slip — that is Petra." No classifier required.
• Rich contextual side-channels. Course comments leak identity through content, not just style: "as the only student who did the field placement in Ghent…", references to a specific question asked in week 6, a disclosed disability, a distinctive project topic. This is textbook singling out.
• First-language and register fingerprints. In internationally diverse European cohorts, L1-transfer patterns and characteristic error signatures are strong, stable cues — and they correlate with nationality and ethnicity, so the re-identification risk lands hardest on the students least protected elsewhere. It compounds the language-bias problem rather than sitting apart from it.

So the academic caveat about short texts and open pools does not reassure the course-evaluation setting — it describes the opposite setting. Shrink the candidate pool to a class and hand the text to the instructor, and you have turned Narayanan''s hard problem into an easy one.

What GDPR actually requires — and why "we removed the names" is not a defence

European institutions cannot treat this as a purely ethical nicety, because it bears directly on whether the data is personal data at all. GDPR Recital 26 says that to decide whether someone is identifiable, "account should be taken of all the means reasonably likely to be used, such as singling out, either by the controller or by another person." The test is not "did we delete the name." It is: given available means — including a lecturer''s knowledge of their own class — is identification reasonably likely?

The former Article 29 Working Party''s Opinion 05/2014 on anonymisation techniques sharpened this: true anonymisation must defeat not just naming but singling out, linkability, and inference. Free-text comments in a small cohort routinely fail all three — a student can be singled out by a unique circumstance, linked across comments by consistent style, and inferred from content. When that is the case, the comments remain personal data, and pseudonymisation-by-name-removal does not discharge the controller''s obligations. This is the same lesson as our pieces on special-category data in free text and on statistical disclosure control — small numbers and rich text quietly reconstitute identity that a suppression threshold was supposed to remove.

The quieter harm: the chilling effect

Even where no one ever actually identifies an author, the perception that they might is corrosive. If students suspect a lecturer can tell who wrote a critical comment — and in a seminar of a dozen, they are right to suspect it — they self-censor. They soften, they generalise, they withhold the specific, actionable criticism that makes feedback worth collecting. The system then congratulates itself on bland, uniformly positive comments and mistakes silence for satisfaction — the dynamic we describe in What Student Silence Means and the MUM effect. A false promise of anonymity does not just risk exposure; it degrades the data even when no one is exposed, because respondents price in the risk.

What to do about it

You cannot make people write in an unrecognisable voice, and you should distrust any vendor claiming to "anonymise" free text perfectly. But you can manage the risk honestly:

1. Stop promising "anonymous"; promise what you actually deliver. If a lecturer sees verbatim comments from a class of fifteen, tell students that. Accurate expectations beat a promise you cannot keep — the honesty argument in Anonymity, Confidentiality, and GDPR.
2. Report themes, not verbatim quotes, to the people being evaluated — especially for small cohorts. Aggregated thematic summaries carry the signal while stripping the stylistic and contextual fingerprints that raw text preserves.
3. Raise and enforce small-cohort thresholds for free text, not just for ratings. A numeric-suppression rule that ignores open comments is protecting the wrong column.
4. Treat identifiable free text as personal data in your data-protection impact assessment, retention schedule, and access controls — because under Recital 26 it very likely is.

Where Koji fits

The design choice that drives the risk is handing an instructor a pile of raw, attributable comments. Koji for Education is built to break that link. Its automatic thematic analysis aggregates open-text feedback into themes and patterns, so what reaches a lecturer for a small module is a synthesis of what students said, not a transcript that preserves each author''s voice and singling-out details. Quality scoring and structured reporting keep the analysis at the theme and programme level, and its GDPR/AVG-appropriate, EU-based data handling treats free text as the personal data it usually is rather than assuming name-removal ends the obligation. Standardised AI moderation also discourages the identity-leaking tangents ("as the only exchange student who…") that make comments singleable in the first place. The goal is not to pretend anonymity is total — it never is — but to close the easy re-identification paths a raw-comment dump leaves wide open.

The same interview and analysis engine powers the main Koji platform for user and customer research, where "anonymous" open-ended feedback from small, known groups — a beta cohort, a pilot team — carries exactly the same stylometric exposure.

Anonymity in course feedback is not a checkbox you tick by deleting a name. It is a property of the whole pipeline, and in a small class, raw free text almost never has it.