Tobit and Censored Regression for Ceiling-Bounded Course Evaluation

In brief

Course-evaluation scales are notorious for ceiling effects: a large share of students mark 5 out of 5, and the distribution bunches against the top. When you then regress that rating on anything — a teaching change, a section, a cohort — ordinary least squares is biased, because for every student who "would have given a 6 or 7 if the scale allowed it," the recorded 5 understates their real evaluation. The scale has censored their opinion at the boundary. The Tobit model (Tobin, 1958) is the standard fix: it treats each top-coded response as "the true latent rating is at least 5" and estimates the underlying relationship accordingly. Where a descriptive analysis of ceiling effects tells you the pile-up exists, Tobit regression is the tool that lets you model through it and recover effect sizes a naive regression shrinks toward zero.

What the research says

James Tobin introduced the model in 1958 (Econometrica) to analyse household spending on durable goods, where many households spend exactly zero — a corner solution that clusters observations at a boundary. Tobin's insight, later dubbed the "Tobit" model, was to posit a latent continuous variable y* generated by a normal linear model, of which we observe only a censored version: below (or above) a threshold we see the threshold value, not the latent one. Estimation by maximum likelihood combines the probability of being at the boundary with the density of the observed values away from it, so the slope coefficients describe the latent relationship rather than the truncated one visible in the raw data.

Amemiya's (1984, Journal of Econometrics) survey formalised the family — distinguishing the standard censored (Type I) Tobit from the many variants — and remains the canonical technical reference. For the education and behavioural-science audience specifically, McBee (2010, Gifted Child Quarterly) is the accessible bridge: titled Modeling Outcomes With Floor or Ceiling Effects: An Introduction to the Tobit Model, it shows that in gifted-education research — where able students routinely top out instruments — ordinary ANOVA and linear regression "produce biased estimates when such effects are present," and demonstrates via simulation that the Tobit model recovers correct inference on a ceiling-affected dataset. That is precisely the situation of a well-liked course whose ratings crowd the top of a 5-point scale. Long's (1997) Regression Models for Categorical and Limited Dependent Variables situates Tobit within the broader limited-dependent-variable toolkit and is the standard graduate text.

The key conceptual point the literature stresses: censoring is a property of the measurement, not the person. Two courses can have identical raw means of 4.6 while their true latent quality differs substantially, because the better course simply has more students jammed against the ceiling where the scale can no longer distinguish them. Tobit is designed to see that hidden gap.

Why it matters for course evaluation in practice

Ceiling effects are not a rare edge case in evaluation; they are the default for popular courses and lenient scales. Three consequences follow:

Real improvements look smaller than they are. If a teaching redesign lifts an already-high course, many students who were at 5 have nowhere to move, so the raw mean barely budges. Tobit, modelling the latent rating, can reveal that the underlying quality shift was real and sizeable.
Differences between strong courses are compressed. Comparing two excellent instructors on raw means is like comparing two sprinters with a stopwatch that stops at the finish line — the scale saturates. Tobit reopens the top end statistically.
Variance and its drivers are distorted. Ceiling censoring shrinks apparent variance and biases any model that depends on it, including some reliability and effect-size calculations. Acknowledging censoring keeps those downstream statistics honest.

For a quality-assurance office, the practical upshot is caution about celebrating — or dismissing — a flat mean on a course that is already near the top. The flatness may be the scale talking, not the teaching.

Limitations and honest caveats

Tobit is the right tool only when its assumptions match the data, and a critical reader will press on them.

Is it really censoring, or just a bounded scale? Tobit assumes a genuine latent variable that the instrument truncates. If students who mark 5 truly mean "exactly the top and no more," there is no latent overflow to recover, and Tobit's premise is wrong. The model is most defensible when there is theoretical reason to believe opinions extend beyond the scale's ceiling.
Normality and homoscedasticity of the latent variable matter. Tobit estimates are more sensitive to violations of normality and constant variance than ordinary regression; when these fail, estimates can be badly biased. Diagnostic checks and, where needed, alternatives (censored quantile regression, two-part models) are essential.
It competes with other bounded-outcome methods. For a rating expressed as a proportion (e.g., percent top-box), beta regression is often the better match; for ordered categories, ordinal regression respects the scale directly. Tobit's niche is a continuous-ish rating censored at a hard boundary — choose deliberately, not by habit.
Interpretation needs care. A Tobit coefficient describes the effect on the latent variable, which is not the same as the effect on the observed, censored mean. Reporting both the latent effect and a marginal effect on the observed outcome avoids overstating the practical size.

How Koji incorporates this

Koji is designed to reduce the censoring problem at the source and to model it honestly when it remains. Because Koji pairs the scale item with an AI-moderated conversational probe, a student who marks the top of the scale is invited to say why and what would have made it even better — recovering, in words, the latent "above the ceiling" signal that a bare 5 discards. That qualitative overflow, run through Koji's automatic thematic analysis, is exactly the information a Tobit model tries to reconstruct statistically, delivered directly. On the quantitative side, Koji's analysis layer is built to recognise skewed, ceiling-bunched distributions rather than reflexively averaging them, so a flat top-heavy mean is flagged for censoring-aware treatment instead of being read at face value. Koji's structured question types also let designers use formats less prone to saturation (best-worst / MaxDiff, comparative judgement) where distinguishing among already-strong courses matters. Koji is designed to mitigate the ceiling-effect blind spot — it does not eliminate the mathematical fact that a bounded scale censors strong opinions, and it never claims to read minds beyond the evidence. Teams running satisfaction or product research beyond education can apply the same conversational engine on Koji's core platform at koji.so.

A worked example

Two lab sections of the same module both average 4.7 on "overall quality." A relieved coordinator calls them equivalent. But section A has 55% of students at the ceiling and section B has 30%. Fitting a Tobit model with a section indicator — treating each 5 as "latent rating at least 5" — estimates a latent-quality gap that the raw means hid: section A's underlying evaluation is meaningfully higher; its students simply ran out of scale. The coordinator now has a defensible reason to study what section A's demonstrator does differently, rather than concluding there is nothing to learn.

Frequently asked questions

How is this different from your article on ceiling effects and skew?

That article diagnoses the problem — it explains what ceiling effects and skew do to descriptive statistics. This one is the modelling response: Tobit regression estimates relationships and effect sizes while accounting for the censoring, rather than just describing the distribution.

When should I use beta regression instead?

When your outcome is naturally a proportion bounded in (0,1) — such as the fraction of top-box responses — beta regression models that directly. Tobit fits a continuous latent rating censored at a hard boundary. If your data are ordered categories, ordinal regression is the cleaner choice.

Does a 5-point Likert item really have a "latent" value above 5?

Only if it is reasonable to believe some students' true regard exceeds what the scale can record. For strongly liked courses that is often plausible. If students genuinely mean "the maximum, full stop," Tobit's censoring premise does not apply and you should not use it.

Will Tobit always give bigger effects than OLS?

Not always, but under ceiling censoring it typically un-shrinks effects that OLS compresses toward zero. The direction and size depend on how much mass sits at the boundary and on the latent-variable assumptions holding.

What diagnostics should accompany a Tobit model?

Check the plausibility of latent normality and constant variance, report the proportion censored, and compare against a censored quantile-regression or two-part alternative. Report both the latent coefficient and a marginal effect on the observed outcome.

Is this only relevant to popular courses?

It is most relevant wherever responses pile against a boundary — the ceiling on lenient scales, but also floors (e.g., "number of complaints" bottoming at zero). The same censoring logic and the same model apply to floor effects.

References

Tobin, J. (1958). Estimation of relationships for limited dependent variables. Econometrica, 26(1), 24–36. https://doi.org/10.2307/1907382
Amemiya, T. (1984). Tobit models: A survey. Journal of Econometrics, 24(1–2), 3–61. https://doi.org/10.1016/0304-4076(84)90074-5
McBee, M. (2010). Modeling outcomes with floor or ceiling effects: An introduction to the Tobit model. Gifted Child Quarterly, 54(4), 314–320. https://doi.org/10.1177/0016986210379095
Long, J. S. (1997). Regression Models for Categorical and Limited Dependent Variables. Sage Publications.

When Ratings Pile Up at 5: Tobit and Censored Regression for Ceiling-Bounded Course Evaluation