Cox Proportional-Hazards Models for Course Evaluation

In brief

A binary "did the student complete the course?" flag throws away the single most informative thing about attrition: its timing. Cox proportional-hazards regression (Cox, 1972) models the instantaneous risk of withdrawal at each moment, conditional on the student having survived until then, as a function of covariates — without assuming any particular shape for the underlying dropout curve. For a quality-assurance office, this turns "12% of the cohort left" into "risk peaked in week 3, and students in the large first-year sections left twice as fast as those in seminars, holding prior attainment constant." That is an actionable evaluation finding; a dropout percentage is not. Cox regression is the standard tool of higher-education retention research, and it belongs in the course-evaluation toolkit for any programme that collects rolling, dated engagement or feedback data.

What the research says

The method comes from David Cox's landmark 1972 paper Regression Models and Life-Tables (Cox, 1972, JRSS-B), one of the most-cited statistics papers ever written. Cox modelled the hazard — the rate at which an event occurs at time t given survival up to t — as a baseline hazard multiplied by an exponential function of covariates: h(t) = h0(t) · exp(β1x1 + ... + βkxk). The decisive move is the proportional-hazards assumption plus a partial likelihood that estimates the covariate effects β without ever specifying h0(t). You get interpretable hazard ratios (a covariate that doubles the hazard means students leave twice as fast at any given moment) while staying agnostic about the shape of the dropout curve itself.

Cox regression sits on top of the older Kaplan–Meier estimator (Kaplan & Meier, 1958, JASA), the nonparametric survival curve that shows the proportion still enrolled over time and correctly handles censoring — students who are still enrolled when the observation window closes contribute information up to that point without being counted as "dropouts." Censoring is exactly why ordinary regression on a completion flag is biased: it treats "still here" and "left" as the only states and ignores how long each student was at risk.

In higher education, the canonical application is Murtaugh, Burns and Schuster (1999, Research in Higher Education), who fitted a proportional-hazards model to 8,867 undergraduates at Oregon State University. They found attrition increased with age and decreased with high-school and first-quarter GPA; non-residents left faster than residents; and — most relevant for evaluation — students who took a Freshman Orientation Course were at reduced risk of dropping out. Crucially, these are statements about timing and rate, derived from dated enrolment records, not from a single end-of-year headcount. Singer and Willett's (2003) textbook Applied Longitudinal Data Analysis is the standard bridge between this survival machinery and the person-period data that education researchers actually hold.

Why it matters for course evaluation in practice

Course evaluation has traditionally been an end-of-term photograph: the students who are still enrolled fill in a survey. But that photograph is taken after the very people whose experience most needs measuring have already gone — and survival analysis is how you study them. Three practical payoffs:

Timing localises the problem. If the hazard of withdrawal spikes in a specific teaching week, that week is where to look — a punishing first assessment, a topic switch, a timetable clash. A completion rate cannot tell you when the course lost people.
Covariates rank the drivers, fairly. A hazard ratio for "large lecture section vs. small seminar," adjusted for prior attainment, isolates the format effect from the intake effect — the same confounding logic behind adjusted-score approaches to ratings.
Early signals become early warnings. With time-varying covariates (attendance, first-assignment submission, an early mid-course pulse score), Cox models can flag rising withdrawal risk while there is still time to intervene — closing the loop rather than autopsying it.

This reframes the evaluation question from "how satisfied were the survivors?" to "who was at risk of leaving, when, and did anything we did change that rate?"

Limitations and honest caveats

Cox regression is powerful but not a licence to claim causation, and it carries assumptions a critical reader will test.

The proportional-hazards assumption can fail. If a covariate's effect grows or shrinks over the term (e.g., a support programme that only helps late), the constant-hazard-ratio assumption is violated. This is checkable — Schoenfeld residuals, log-log plots, or an explicit covariate-by-time interaction — and fixable with stratified or time-varying-coefficient models, but it must be checked, not assumed.
Observational hazards are not causal effects. A hazard ratio showing that large sections lose students faster is confounded by everything that sorts students into large sections. Survival modelling adjusts for measured covariates only; unmeasured confounding remains, exactly as in any observational comparison.
Withdrawal is a competing-risks problem. Leaving for a better opportunity, transferring, and failing out are different events with different meanings; treating them as one "dropout" outcome can mislead. Competing-risks extensions (Fine–Gray subdistribution models) exist and should be used when the distinction matters.
Interval-censoring and small cohorts. If you only know withdrawal happened between two dates, discrete-time survival methods are the better match; and hazard ratios from a single small class will be noisy, so pool across cohorts and report confidence intervals.

Naming these limits is what separates a defensible retention analysis from a dashboard number.

How Koji incorporates this

Koji is built to generate exactly the dated, longitudinal evidence Cox models need, rather than a single end-of-term snapshot. Because Koji runs AI-moderated conversational check-ins across the teaching cycle — not just one final survey — each student contributes a time-stamped trail of engagement and sentiment that forms a natural person-period dataset. Koji's structured questions (scale, single_choice, yes_no, open_ended) attached to mid-cycle pulses supply candidate time-varying covariates, and its automatic thematic analysis of open-text turns a rising withdrawal hazard into a reason, not just a rate — the qualitative "why" behind the week-3 spike. Koji's quality scoring and bias-aware reporting are designed to keep the covariate side honest, so a hazard ratio reflects the course rather than an artefact of who answered. The platform is designed to mitigate the survivor-only blind spot of traditional evaluation by capturing signal from students before they disengage — it does not eliminate attrition or prove causation, but it moves the evidence upstream to where intervention is still possible. Teams running broader retention or product-lifecycle research can apply the same AI-moderated interview engine on Koji's core platform at koji.so.

A worked example

Imagine a first-year statistics module with a 15% withdrawal rate. A completion flag stops there. A Kaplan–Meier curve instead shows enrolment holding flat until week 3, then dropping sharply after the first quiz. A Cox model with covariates for section size, prior maths grade, and early-quiz score yields: prior grade hazard ratio 0.6 (each grade band up cuts withdrawal risk 40%), large-section hazard ratio 1.9 (nearly double the risk), and — after adjustment — a non-significant effect of the instructor. The evaluation conclusion writes itself: the module is not losing students because of who teaches it but because a high-stakes early quiz in oversized sections is filtering out weaker-prepared students. That is a curriculum-design finding a satisfaction mean would never surface.

Frequently asked questions

How is this different from your discrete-time survival analysis of survey breakoff?

That method models where students abandon the questionnaire itself — a within-survey behaviour measured in questions. Cox regression here models withdrawal from the course over calendar time. Same survival logic, entirely different event and time scale.

Isn't this just the same as our differential-attrition analysis?

No. Differential-attrition analysis asks whether the students who dropped out differ from those who stayed, threatening a comparison. Cox regression models the rate and timing of leaving and lets you attach explanatory covariates to it. They are complementary: attrition analysis protects your other comparisons; survival analysis studies the leaving itself.

Do we need exact withdrawal dates?

For continuous-time Cox regression, yes — you need the time-to-event. If you only know withdrawal occurred within a teaching week or between two pulses, use discrete-time (interval-censored) survival methods instead; the interpretation is nearly identical.

What is a hazard ratio, in plain terms?

It is the multiplicative change in the moment-to-moment risk of leaving associated with a covariate. A hazard ratio of 2 means "students with this characteristic leave twice as fast at any given point"; 0.5 means "half as fast." It is not a probability and not a risk over the whole term.

Can a hazard ratio prove a teaching change reduced dropout?

Not on its own. It shows association after adjusting for measured covariates. To argue causation you need a design — a difference-in-differences or interrupted-time-series comparison around the change — layered on top of the survival outcome.

How small a cohort is too small?

There is no hard cutoff, but hazard-ratio estimates from a single class of, say, 20 students will have very wide confidence intervals. Pool across cohorts or terms, report the intervals, and resist reading precise ratios off tiny samples.

References

Cox, D. R. (1972). Regression models and life-tables. Journal of the Royal Statistical Society: Series B (Methodological), 34(2), 187–220. https://doi.org/10.1111/j.2517-6161.1972.tb00899.x
Kaplan, E. L., & Meier, P. (1958). Nonparametric estimation from incomplete observations. Journal of the American Statistical Association, 53(282), 457–481. https://doi.org/10.1080/01621459.1958.10501452
Murtaugh, P. A., Burns, L. D., & Schuster, J. (1999). Predicting the retention of university students. Research in Higher Education, 40(3), 355–371. https://doi.org/10.1023/A:1018755201899
Singer, J. D., & Willett, J. B. (2003). Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence. Oxford University Press. https://doi.org/10.1093/acprof:oso/9780195152968.001.0001

When, Not Just Whether, Students Leave: Cox Proportional-Hazards Models for Course Evaluation