Small-Class Course-Evaluation Reporting: Cell Suppression, k-Anonymity, Differential Privacy

In brief

When a course has only a handful of respondents, publishing an average, a breakdown, or a verbatim comment can quietly expose individuals — the instructor can often infer who gave the low score, and a demographic cross-tab can single out the one international student in the room. Statistical disclosure control is the discipline that manages this. Its tools range from simple minimum-cell suppression (report nothing below a threshold, and suppress complementary cells so the total does not leak the hidden one), through k-anonymity (never let a record be less distinguishable than k-1 others), to differential privacy (add mathematically calibrated noise so that any single student's participation is provably hard to detect). The unavoidable tension is that every one of these methods trades away accuracy, and on already-tiny classes that trade is steep — often the honest answer is to report less, not more.

What the research says

The re-identification risk is not hypothetical. Sweeney (2002), in the International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, formalised k-anonymity: a release satisfies k-anonymity if every record is indistinguishable from at least k-1 others on the "quasi-identifiers" (attributes like programme, year, gender, nationality) that could be linked to outside information. Her companion work demonstrated that small combinations of such attributes uniquely identify most people — the classic result that date of birth, sex and postal code alone identify a large majority of a population. In a course-evaluation table broken down by demographic subgroup, the same logic means a single respondent in a cell is effectively named.

Simple cell suppression — hiding any count below a threshold — is the traditional education-sector response, codified in guidance such as the U.S. Department of Education Privacy Technical Assistance Center's disclosure-avoidance materials. But suppression is leakier than it looks: if you publish a group total and suppress one small subgroup, the suppressed number can be recovered by subtraction, which is why complementary suppression of additional non-sensitive cells is often required.

Differential privacy, introduced by Dwork and colleagues (2006) and synthesised in Dwork and Roth's (2014) monograph The Algorithmic Foundations of Differential Privacy, offers a stronger, quantitative guarantee. Rather than editing the output after the fact, it adds noise calibrated to a statistic's sensitivity — how much one person's data could change it — governed by a privacy-loss parameter (epsilon). The formal promise is that the published result is almost the same whether or not any given student participated, so an adversary cannot confidently learn that a particular student responded at all. The literature is candid that this noise-based approach generally dominates naive suppression on the privacy-utility frontier, but that the guarantee costs accuracy, and that small samples force a harsh choice between a meaningful epsilon and a usable number.

Why it matters for course evaluation in practice

Every quality office faces the small-n problem. A seminar of six, an elective with four responses, a demographic filter that leaves one student — each is a disclosure risk, and the risk is sharpest for exactly the sensitive, free-text feedback that is most valuable. A student who writes a pointed comment in a class of five has little protection if the verbatim is shown to the instructor with a subgroup label attached.

This is why a minimum-response threshold before any result is shown is the single most important reporting rule, and it connects to the reliability argument in how many responses you need: below a floor, a number is neither statistically trustworthy nor safe to publish. It also interacts with how you report. Choices like reporting a percentage top-box instead of a mean can either help or hurt — a "100% favourable" on a class of three still discloses that all three were positive. And funnel plots, which explicitly widen the uncertainty band for small classes, are a disclosure-aware way to show that a tiny cohort's score simply cannot be pinned down. The privacy case reinforces the statistical one: small cells should be aggregated up, not drilled into.

Protecting anonymity is also not merely defensive — it changes the data. The honesty of responses depends on students trusting that they cannot be identified, the same mechanism behind work on mode effects and social desirability and the randomized response technique for sensitive items. Weak disclosure control does not just risk exposure; it suppresses candour.

Limitations and honest caveats

A rigorous reader will push on several points.

There is no free lunch. Every disclosure-control method degrades accuracy. Differential privacy makes the trade explicit and tunable, but on a class of five even a modest privacy guarantee can swamp the signal entirely. Sometimes the correct, honest output is "too few responses to report."

Thresholds are necessary but not sufficient. A minimum-n rule stops the most obvious exposure, but differencing attacks — combining multiple released tables, or comparing this year to last — can still reconstruct suppressed cells. Complementary suppression and consistent rules across releases are needed, and are easy to get wrong.

Differential privacy is hard to deploy well. Choosing epsilon is a policy decision with no purely technical answer; a large epsilon offers little real protection while sounding rigorous, and the cumulative privacy loss across many queries must be tracked (the "privacy budget"). It is a guarantee about a mechanism, not a magic setting.

Qualitative data resists these tools. k-anonymity and differential privacy are built for structured tables and counts. A free-text comment can self-identify through content and voice no matter what the surrounding statistics do; redaction and careful routing, not noise addition, are the relevant safeguards there.

Aggregation has its own costs. Rolling small classes up into a department protects individuals but can hide a genuinely struggling course. Disclosure control and actionable feedback are in real tension, and the balance is a judgement, not a formula.

How Koji incorporates this

Koji treats respondent protection as a precondition for trustworthy data, not a compliance afterthought, and several of its mechanisms map onto the methods above. The platform is designed to enforce minimum-response thresholds before results are displayed, so a class below a safe floor does not surface an average, a breakdown, or an attributable comment in the first place — the single most effective disclosure control. Reporting defaults toward aggregated, programme-level views rather than drill-downs into cells small enough to expose an individual, which is the same altitude at which its statistics become reliable.

For the open text that resists tabular methods, Koji's automatic thematic analysis is designed to report patterns at the theme level rather than routing raw, potentially self-identifying verbatims to an instructor who could recognise the author, and its AI-moderated conversational interviews collect candid input under an anonymity framing that the disclosure literature shows is essential to honesty. Because Koji handles European higher-education data, this sits alongside GDPR data-minimisation obligations, where collecting and exposing less identifiable detail is not only safer statistically but expected legally.

We frame these as protections designed to reduce re-identification risk, not as a formal differential-privacy guarantee on every report — deploying calibrated-noise mechanisms responsibly, including setting a defensible privacy budget, remains an institutional policy decision. Koji's core research platform at koji.so applies the same AI-moderated interview engine and the same minimum-cohort, aggregate-first reporting discipline to product and customer research, where protecting small respondent groups is an identical concern.

Related resources

Frequently asked questions

Why is reporting an average for a very small class a privacy problem, not just a reliability one?

Because with only a few respondents an average, a subgroup breakdown, or a verbatim comment can reveal who said what. An instructor can often infer which student gave a low score, and a demographic filter that leaves a single respondent effectively names that person. Small cells are both statistically unstable and disclosive.

What is the simplest effective safeguard?

A minimum-response threshold: do not display any result — mean, breakdown, or comment — until a course has at least a set number of responses. It is the single most effective disclosure control and it aligns with the reliability floor below which a number should not be trusted anyway.

What is k-anonymity in this context?

k-anonymity requires that every record be indistinguishable from at least k-1 others on the quasi-identifiers that could be linked to outside information, such as programme, year, gender, and nationality. In an evaluation table it means never letting a demographic cross-tab isolate a single student; achieving it usually requires generalising or suppressing categories.

How is differential privacy different from just suppressing small cells?

Suppression hides outputs after the fact and can leak through subtraction of published totals. Differential privacy instead adds noise calibrated to how much one person could change a statistic, giving a quantitative guarantee that a result is almost the same whether or not any given student participated. It generally protects better but costs accuracy and requires choosing a privacy-loss parameter.

Can these methods protect free-text comments?

Not directly. k-anonymity and differential privacy are built for counts and tables; a comment can self-identify through its content and voice regardless of the surrounding statistics. Protecting open text relies on theme-level reporting, redaction, and careful routing rather than noise addition or cell suppression.

Does aggregating small classes hide real problems?

It can. Rolling a small course up into a department protects individuals but may mask a genuinely struggling class. Disclosure control and actionable, granular feedback are in tension, so the level of aggregation is a judgement that balances student protection against the usefulness of the report.

References

Sweeney, L. (2002). k-anonymity: A model for protecting privacy. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 10(5), 557-570. https://doi.org/10.1142/S0218488502001648
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006). Calibrating noise to sensitivity in private data analysis. Theory of Cryptography Conference (TCC), LNCS 3876, 265-284. https://doi.org/10.1007/11681878_14
Dwork, C., & Roth, A. (2014). The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4), 211-407. https://doi.org/10.1561/0400000042
U.S. Department of Education, Privacy Technical Assistance Center. (2016). Frequently Asked Questions — Disclosure Avoidance. https://studentprivacy.ed.gov/resources/frequently-asked-questions-disclosure-avoidance

Reporting a Class of Four Without Exposing Anyone: Cell Suppression, k-Anonymity, and Differential Privacy