Splines and GAMs for Non-Linear Course-Evaluation Analysis

In brief

Most course-evaluation analyses that relate a rating to a continuous predictor — workload, difficulty, class size, contact hours — quietly assume the relationship is a straight line: every extra hour of workload changes satisfaction by the same amount, forever. That assumption is usually wrong. The evidence on workload and difficulty points to non-monotone, curved relationships (too little challenge bores, too much overwhelms, with an optimum in between). Restricted cubic splines and generalized additive models (GAMs) let you estimate that curve directly from the data — flexible where the relationship bends, stable at the extremes — instead of either forcing a line or, worse, chopping the predictor into "low/medium/high" bins that destroy information and invite bias. If your quality dashboard reports "workload is negatively correlated with satisfaction," a spline may reveal that is only true past a threshold, and that the real story is an inverted-U.

What the research says

The statistical foundations are well established. Cubic splines fit a smooth curve by joining low-order polynomial pieces at chosen points called knots, constrained to meet smoothly. Durrleman and Simon (1989, Statistics in Medicine) brought restricted (natural) cubic splines into applied research, showing how they capture non-linear covariate effects while the "restriction" — forcing the curve to be linear beyond the outermost knots — tames the wild tail behaviour that plagues ordinary polynomials. Their worked comparison against nonparametric fits on survival data demonstrated that a handful of knots recovers the true shape without overfitting.

Hastie and Tibshirani (1986, Statistical Science) generalised this into the generalized additive model: an outcome modelled as a sum of smooth functions of each predictor, g(E[y]) = β0 + f1(x1) + f2(x2) + ..., where each f is estimated nonparametrically (via splines or local regression) and the link function g lets the same framework handle bounded, count, or binary outcomes. GAMs keep the interpretability of additive models — you can plot each predictor's estimated curve holding the others fixed — while dropping the linearity straitjacket. Modern implementations choose smoothness automatically by penalising wiggliness, so the analyst does not hand-pick how curvy the fit should be.

The complementary, and equally important, finding is what not to do instead. Royston, Altman and Sauerbrei (2006, Statistics in Medicine) titled their paper bluntly: dichotomising continuous predictors is "a bad idea." Cutting workload at its median to compare "high vs. low" throws away information, loses statistical power, assumes a step-change at the cutpoint that rarely exists, and — when the cutpoint is chosen to maximise significance — manufactures bias. Splines and GAMs are the constructive answer to that critique: keep the variable continuous, let the shape emerge. Harrell's (2015) Regression Modeling Strategies is the standard practitioner reference, recommending restricted cubic splines with 3–5 knots as a sensible default for most applied models.

Why it matters for course evaluation in practice

Three things go wrong when evaluators assume linearity or bin their predictors:

You miss the optimum. The pedagogically interesting result is almost always a turning point — the workload or difficulty level at which satisfaction (or self-reported learning) peaks. A linear model reports only a slope and, by construction, cannot find a peak. A spline shows it on a graph a dean can read.
You get the sign wrong at the ends. If the true relationship is an inverted-U, a straight line averages the rising and falling halves into a near-zero slope, and you conclude "workload does not affect ratings" when in fact it strongly does — just non-monotonically.
Binning invents cliffs and hides gradients. "High vs. low difficulty" implies satisfaction is flat within each bin and jumps at the boundary. Splines replace that fiction with the actual gradient, which is what curriculum designers need to set the right level.

For programme-level quality work, a spline plot of rating against class size — showing, say, a flat relationship up to ~40 students and a steep decline thereafter — is directly decision-relevant in a way a single regression coefficient never is.

Limitations and honest caveats

Flexibility has costs, and a sophisticated reader will probe them.

Splines fit noise if under-penalised. With too many knots or too little smoothing, a GAM will chase random wiggles in a small class's data and "discover" curves that are artefacts. Penalised smoothing and cross-validation guard against this, but small samples still demand restraint — few knots, and honest confidence bands.
Correlation, not causation. A curved workload–rating relationship is still observational. Students self-select into courses; unmeasured confounders shape both workload perception and satisfaction. A spline describes the shape of an association, not a causal dose–response.
Interpretation is graphical, not a single number. You cannot summarise a spline with one tidy coefficient, which makes reporting to committees harder and tempts people back to the line. The discipline is to report the plot with its uncertainty band and describe the shape in words.
Extrapolation is dangerous. Restricted cubic splines are linear beyond the outer knots by design; trust the curve only within the range of observed data, and never read predictions off the extreme tails of a sparse predictor.

How Koji incorporates this

Koji is designed to produce the kind of rich, continuous, multi-signal data on which splines and GAMs actually pay off — and to keep the interpretation honest. Because Koji collects structured scale and single_choice items alongside AI-moderated open-text, an evaluation can carry genuine continuous predictors (perceived workload, hours studied, pace) rather than only a coarse "too much / about right / too little" tick-box, giving a spline something real to bend around. Koji's analysis layer is built to model relationships rather than just tabulate averages, so a non-monotone workload–satisfaction curve surfaces as a finding instead of being flattened into a misleading correlation, and its bias-aware reporting is designed to discourage the median-split shortcut that Royston and colleagues warn against. When the conversational engine detects that a student sits near the bottom or top of a predictor range — the very regions where a curve turns — Koji's follow-up probes can ask why, adding the qualitative texture a spline plot alone cannot supply. Koji is designed to mitigate the linear-and-binned habits of legacy evaluation tools; it does not turn observational shape into proof of cause. Teams doing product or customer research can apply the same engine on Koji's core platform at koji.so.

A worked example

A department believes "harder courses score worse" and reports a modest negative correlation between average difficulty rating and satisfaction. Refitting satisfaction on difficulty with a restricted cubic spline (four knots) tells a different story: satisfaction rises with difficulty up to a moderate level, plateaus, then falls only for the most punishing courses — a classic inverted-U consistent with desirable-difficulty theory. The linear coefficient was the average of a rising and a falling arm, near zero and misleading. The spline hands the curriculum committee a concrete target: the difficulty band where satisfaction and challenge co-peak, and the threshold past which they should not push.

Frequently asked questions

How is this different from quantile or ordinal regression, which you also recommend?

Those change what you model — quantile regression models the tails of the rating distribution; ordinal regression respects the ordered-category scale. Splines and GAMs change the shape of the predictor's effect, letting it curve. They are complementary and can be combined (e.g., a spline term inside an ordinal or quantile model).

Isn't regression discontinuity also about non-linear effects at a point?

No. Regression discontinuity exploits a known threshold in a running variable to estimate a causal jump. Splines estimate a smooth curve everywhere with no assumed break and make no causal claim. Different questions, different designs.

How many knots should I use?

Harrell's default of 3–5 restricted-cubic-spline knots handles most course-evaluation relationships. More knots rarely help and risk overfitting; placement at data quantiles is standard. Let penalised smoothing or cross-validation fine-tune rather than hand-tuning.

Why not just split workload into low/medium/high — it's easier to explain?

Because it loses power, assumes non-existent step changes at the cut points, and — if the cuts are chosen to look significant — biases the result. Royston, Altman and Sauerbrei (2006) show the costs directly. A spline plot is only marginally harder to explain and far more truthful.

Can I use a GAM when my outcome is a bounded rating or a proportion?

Yes. GAMs take a link function, so you can pair smooth predictor terms with a distribution suited to bounded, count, or binary outcomes — the same reason we recommend beta regression for proportions and count models for comment counts.

Do small classes make splines unreliable?

They can. With few students the curve is poorly identified and confidence bands are wide. Use fewer knots, pool across sections where sensible, and always show the uncertainty band so readers do not over-read wiggles.

References

Hastie, T., & Tibshirani, R. (1986). Generalized additive models. Statistical Science, 1(3), 297–318. https://doi.org/10.1214/ss/1177013604
Durrleman, S., & Simon, R. (1989). Flexible regression models with cubic splines. Statistics in Medicine, 8(5), 551–561. https://doi.org/10.1002/sim.4780080504
Royston, P., Altman, D. G., & Sauerbrei, W. (2006). Dichotomizing continuous predictors in multiple regression: a bad idea. Statistics in Medicine, 25(1), 127–141. https://doi.org/10.1002/sim.2331
Harrell, F. E. (2015). Regression Modeling Strategies (2nd ed.). Springer. https://doi.org/10.1007/978-3-319-19425-7

Is the Workload-Rating Relationship a Straight Line? Splines and Generalized Additive Models for Course Evaluation