Your Outcome Is a Proportion, Not a Mean: Beta Regression for Course Evaluation

In brief

When a course-evaluation outcome is a proportion bounded between 0 and 1 — the fraction of a class who would recommend the course, the top-box rate on an item, the response rate itself — ordinary linear regression is the wrong instrument: it can predict impossible values below 0 or above 1, and it assumes a constant error variance that a bounded proportion cannot have. Beta regression, introduced by Silvia Ferrari and Francisco Cribari-Neto in 2004, models the outcome with the flexible beta distribution, links the mean through a logit so predictions stay inside (0,1), and lets the variance shrink naturally near the boundaries. For a quality office that reports and compares rates, it is a better-fitting, more honest alternative to averaging or to a linear model on a percentage.

What the research says

Ferrari and Cribari-Neto (2004, Journal of Applied Statistics 31(7):799-815, doi:10.1080/0266476042000214501) proposed a regression model for a response that is a rate or proportion in the open interval (0,1). Their key moves are two. First, they reparameterise the beta distribution in terms of a mean (mu) and a precision parameter (phi), so the model reads like a familiar generalised linear model. Second, they connect the mean to predictors through a link function — typically the logit — guaranteeing fitted values remain between 0 and 1. Crucially, the beta distribution is heteroscedastic by construction: its variance is mu(1 minus mu) / (1 plus phi), which is largest when the mean sits near 0.5 and smallest near the boundaries. This matches the reality of proportions, where a rate near 90% simply cannot vary as much as one near 50%. A linear model that assumes constant variance gets this exactly wrong.

Michael Smithson and Jay Verkuilen (2006, Psychological Methods 11(1):54-71, doi:10.1037/1082-989X.11.1.54) brought beta regression into the social sciences with the memorably titled A Better Lemon Squeezer?. Their contribution matters for practitioners in two ways. They showed how modelling the precision phi with its own predictors captures the fact that some groups produce more consistent proportions than others — dispersion is itself a finding. And they supplied the standard fix for boundary values: because the beta distribution is defined on the open interval, exact 0s and 1s must be nudged inward, using the transformation (y(n minus 1) plus 0.5) / n, where n is the sample size. Francisco Cribari-Neto and Achim Zeileis (2010, Journal of Statistical Software 34(2), doi:10.18637/jss.v034.i02) then made the method routine with the betareg package, including variable-dispersion models and diagnostic tools.

For the not-uncommon case where genuine 0s and 1s occur — a course no one recommended, or an item everyone top-boxed — the literature offers zero-one-inflated beta regression (Ospina and Ferrari, 2012, Computational Statistics and Data Analysis 56(6):1609-1623, doi:10.1016/j.csda.2011.10.005), a mixture that models the exact boundary events with their own process and the interior proportions with a beta component.

Why it matters for course evaluation in practice

Quality offices are awash in proportions, and they usually mistreat them. A top-box rate — the share choosing the highest option — is a proportion. A "would recommend" percentage, the cousin of a Net Promoter score, is a proportion. A response rate is a proportion. When an analyst regresses one of these on class size, discipline or modality with ordinary least squares, three things go wrong: fitted values can leave the legal range, standard errors are miscalibrated because the variance is not constant, and tests are distorted. Beta regression removes all three problems in one model.

The practical payoffs are concrete. Honest comparison of rates across unequal groups. A large lecture and a small seminar with the same 80% recommend rate carry very different uncertainty; beta regression encodes that through the precision parameter instead of pretending both are equally firm. Modelling the spread, not just the level. Because phi can depend on predictors, you can ask which courses produce erratic proportions — a variable-dispersion result that a mean-only model cannot see. Better small-sample behaviour near ceilings. Course evaluation is notorious for scores bunched near the top; a proportion at 92% modelled linearly will generate nonsense predictions above 100% for favourable subgroups, while beta regression stays bounded. It sits alongside, not against, the ordinal and count models this knowledge base already covers: ordinal regression is for a single Likert response, count models for tallies, and beta regression for a proportion aggregated from many responses.

Limitations and honest caveats

Beta regression is the right tool for a narrow, well-defined job, and misapplying it is easy. It is for continuous proportions, not for a count of successes out of a known denominator. If you have 18 of 25 students recommending, that is binomial data and logistic regression is the correct model; beta regression is for a proportion treated as continuous (an aggregate rate, an index, a fraction with an ill-defined denominator). Boundary handling is a modelling choice, not a neutral step. The (y(n minus 1) plus 0.5)/n squeeze and the zero-one-inflated alternative can give different answers when many observations sit exactly at 0 or 1, and an analyst should report which was used and why. Coefficients are on the logit scale. As with logistic regression, effects are not read directly as percentage-point changes; they need to be transformed to be interpretable, and a reader who skips this will overstate effect sizes. It models the outcome you feed it, biases and all. If the top-box rate is inflated by leniency or by the response-quality problems documented elsewhere here, beta regression will faithfully model an inflated proportion. And with very small classes, precision estimation is unstable — the same small-cohort caution that applies to every course-evaluation statistic applies here.

How Koji incorporates this

Koji reports proportion-type outcomes — recommend rates, top-box shares, the fraction of a cohort raising a given theme — and its analytics are designed to model them as bounded quantities rather than as means on a scale that has no natural zero. When Koji compares such a rate across courses, cohorts or delivery modes, the underlying model is designed to keep predictions inside (0,1) and to let uncertainty widen toward the middle of the range and tighten near the boundaries, exactly as a beta specification does, so a 90% recommend rate from a small seminar is not reported with the same false firmness as a 90% rate from a large lecture. Because dispersion can be modelled explicitly, Koji can surface which courses produce inconsistent proportions, not merely which score lower on average — a signal for a programme director that the experience is uneven rather than uniformly weak. Where genuine boundary cases arise — a unanimous cohort — Koji is reporting is built to flag the exact-0 or exact-1 event rather than silently squeezing it, keeping the analyst in control of the modelling choice. And because a proportion inherits every bias of the responses it aggregates, Koji pairs rate modelling with its bias-aware and quality-scoring layers rather than treating a clean-looking percentage as the end of the story. Koji is core research platform at koji.so applies the same bounded-outcome reasoning to product and customer metrics such as recommend rates and feature-adoption proportions.

The takeaway for an analyst is disciplinary: the moment your outcome is a proportion, the mean is the wrong summary and ordinary regression is the wrong model. Beta regression respects the boundaries and the shape of the data, and it turns "the spread of the rate" from a nuisance into a reportable finding.

Frequently asked questions

When should I use beta regression instead of logistic regression?

Use logistic regression when you have a count of successes out of a known number of trials (18 of 25 students). Use beta regression when the outcome is a continuous proportion — an aggregate rate, an index scaled to (0,1), or a fraction whose denominator is not a clean count of Bernoulli trials.

What do I do about proportions that are exactly 0 or 1?

Beta regression is defined on the open interval, so exact boundary values need handling. The common fix is the Smithson-Verkuilen squeeze, (y(n minus 1) plus 0.5)/n; when many observations sit exactly at 0 or 1, a zero-one-inflated beta model that treats those events explicitly is more honest. Report whichever you used.

How is beta regression different from just modelling the mean rating?

A mean rating is an average on an ordinal scale; a proportion (like top-box or recommend rate) is bounded in (0,1) with a variance that changes across the range. Beta regression models that bounded, heteroscedastic quantity directly, whereas averaging or linear regression assumes an unbounded outcome with constant variance.

Can beta regression tell me about the variability of a proportion, not just its level?

Yes. A variable-dispersion beta model lets the precision parameter depend on predictors, so you can identify which courses or cohorts produce erratic versus consistent proportions — a finding a mean-only analysis cannot produce.

Does it fix ceiling effects in course evaluations?

It handles the statistical consequence of proportions bunched near a boundary — impossible predictions and mis-estimated variance — but it does not remove the substantive ceiling. If almost everyone top-boxes, the instrument may lack headroom to discriminate, which is a design issue beyond any model.

Is beta regression interpretable for non-statisticians?

Its coefficients live on the logit scale, so they need transforming into predicted proportions or odds-style statements before a committee reads them. Presented as fitted rates for representative cases, it is very interpretable; presented as raw coefficients, it is easy to misread.

References

• Ferrari, S. L. P., & Cribari-Neto, F. (2004). Beta regression for modelling rates and proportions. Journal of Applied Statistics, 31(7), 799-815. doi:10.1080/0266476042000214501
• Smithson, M., & Verkuilen, J. (2006). A better lemon squeezer? Maximum-likelihood regression with beta-distributed dependent variables. Psychological Methods, 11(1), 54-71. doi:10.1037/1082-989X.11.1.54
• Cribari-Neto, F., & Zeileis, A. (2010). Beta regression in R. Journal of Statistical Software, 34(2), 1-24. doi:10.18637/jss.v034.i02
• Ospina, R., & Ferrari, S. L. P. (2012). A general class of zero-or-one inflated beta regression models. Computational Statistics and Data Analysis, 56(6), 1609-1623. doi:10.1016/j.csda.2011.10.005

Related Resources

• Top-Box vs Mean: How to Report Course Evaluation Scores Without Throwing Away Information
• Ordinal Regression for Course-Evaluation Data: Why Cumulative-Link Models Beat Averaging Likert Scores
• Count Models for Course Evaluation: Why You Should Not Average the Number of Comments
• Should You Use Net Promoter Score for Courses? The "Would You Recommend" Question in Higher Education
• The Average Hides the Tails: Quantile Regression for Course-Evaluation Data
• How Many Responses Do You Need for a Reliable Course Evaluation?