Latent Transition Analysis for Course Evaluation: Modelling Segment Change

In brief

Latent transition analysis (LTA) is the longitudinal extension of latent class or latent profile analysis: instead of describing the types of students who respond to a course evaluation at one point in time, it estimates the probability that a student in one segment at an early wave moves to a different segment at a later wave. For course evaluation, its most useful output is a transition matrix — for example, the proportion of "disengaged at mid-semester" students who become "recovered and satisfied" by the end — which turns two static snapshots into a picture of change. LTA is well validated in the social and health sciences, but it is demanding of data, sensitive to how you fix the measurement model, and easy to over-interpret with the small cohorts typical of a single course.

What the research says

LTA belongs to the family of finite-mixture models. Its foundational reference is Collins and Lanza (2010), whose monograph unifies latent class analysis (LCA, for categorical indicators), latent profile analysis (LPA, for continuous indicators), and their longitudinal extension into a single estimation framework. The core idea is that a small number of unobserved "statuses" (segments) underlie the observed response pattern at each occasion, and a Markov process governs how individuals move between those statuses across occasions. The model returns three things at once: the item-response profiles that define each status, the prevalence of each status at each wave, and a transition-probability matrix linking waves.

Lanza and Collins (2008) demonstrated the approach on longitudinal data from college students, modelling movement between latent statuses of dating and sexual-risk behaviour over time and showing how transition probabilities can be predicted by covariates — a template directly transferable to how student segments shift across an academic term. More recently, Nylund-Gibson and colleagues (2023), in "Ten Frequently Asked Questions About Latent Transition Analysis" (Psychological Methods), give practical guidance on the decisions that most affect results: how many statuses to extract, whether to impose measurement invariance (forcing the meaning of a status to stay constant across waves so that "movement" reflects real change rather than a shifting yardstick), sample-size expectations, and how to bring in auxiliary variables without distorting the class solution.

The empirical signal these papers share is that aggregate means often hide compositional change. Two evaluation waves can show an identical average while masking substantial churn — some students deteriorating, others recovering — and the transition matrix is what exposes it.

Why it matters for course evaluation in practice

Most evaluation reporting compares averages: the mid-semester mean versus the end-of-semester mean. That comparison is blind to who moved where. Suppose a mid-cycle pulse identifies three segments — "struggling with pace," "engaged but critical of assessment," and "broadly satisfied." An LTA on the paired mid-and-end responses answers questions a mean cannot: Did the "struggling with pace" group mostly recover after the instructor slowed down, or did they disengage further and stop attending? Did an assessment redesign convert critics into satisfied students, or simply move satisfied students into criticism?

This reframes course evaluation from a scoreboard into a diagnostic of change trajectories. It complements segment-based cross-sectional work such as latent profile analysis, and it pairs naturally with in-semester collection designs like experience sampling and the consultation-driven improvements documented in the mid-semester feedback meta-analyses. Where a low-scoring class "improves" the next year for no real reason, regression to the mean is the usual culprit; LTA is one of the few tools that can distinguish genuine within-student movement from noise, because it models each student's status at both waves rather than the class average.

Limitations and honest caveats

A PhD reader will raise several objections, and they are right to.

Sample size. Mixture models are hungry. Estimating stable statuses and a transition matrix requires far more respondents than a single course typically yields; guidance in the LTA literature points to samples in the hundreds, not the twenty or thirty paired responses a seminar produces. For a single small course, LTA will over-fit or fail to converge. It is a programme- or cohort-level tool, pooling many sections, not a per-seminar one.

Class enumeration is a judgement call. The number of statuses is chosen by the analyst using fit indices (BIC, the bootstrap likelihood-ratio test) plus interpretability. Different defensible choices yield different transition stories — the same forking-paths problem addressed by specification-curve analysis. Report the enumeration decision transparently.

Measurement invariance is an assumption, not a fact. If you do not constrain the status definitions to be equivalent across waves, an apparent "transition" may just reflect the segments meaning something different at the end of term. Invariance should be tested, not assumed.

Attrition is a confound. Students who drop out between waves are rarely missing at random; differential attrition can manufacture spurious transitions. LTA can incorporate covariates and missing-data handling, but it cannot rescue a design where the disengaged simply vanish from the end-of-term wave.

Statuses are models, not people. A latent status is a statistical abstraction with classification uncertainty; assigning students to their most-likely status and then treating that label as certain understates the error.

How Koji incorporates this

Koji is built around collecting evaluation evidence more than once, which is the precondition LTA needs. Because Koji supports mid-cycle and formative collection alongside end-of-term summative runs, the platform naturally generates the paired, multi-wave, student-linked data that a transition model requires — rather than the single end-of-term snapshot that makes LTA impossible.

Concretely, Koji is designed to support this analysis in several ways. Its AI-moderated conversational interviews probe beyond a single Likert number, and its structured question types (open_ended, scale, single_choice, multiple_choice, ranking, yes_no) yield the multivariate indicator set that defines meaningful segments in the first place. Automatic thematic analysis of open-text responses can supply qualitative indicators — a "struggling with pace" status is far more credible when the numeric profile is corroborated by the language students use. Because collection is designed to run across the term rather than only at the end, Koji preserves the wave structure and the within-student linkage that a transition matrix depends on. And its closing-the-loop action tracking means a transition can be tied to an intervention: if the instructor changed the pacing after a mid-cycle pulse, the model can ask whether the "struggling" segment actually moved.

We frame this as designed to support segment-transition analysis at programme scale, not as a guarantee that any single course has enough data to fit one. Koji's core research platform at koji.so applies the same AI-moderated interview engine to product and customer research, where the same longitudinal-segmentation logic — how a cohort of users moves between behavioural states over time — is a standard analytical need.

Related resources

Frequently asked questions

How is latent transition analysis different from latent profile analysis?

Latent profile analysis describes the segments present at a single point in time. Latent transition analysis takes the same idea but adds a time dimension: it estimates the probability that a student in one segment at an early wave moves to a different segment at a later wave. The distinctive output is a transition-probability matrix, which LPA cannot produce because it has only one occasion.

How much data do I need to run an LTA on course evaluations?

Far more than a single course provides. Mixture models estimate both the segment definitions and the movement between them, which typically requires samples in the hundreds. Treat LTA as a programme- or cohort-level tool that pools many sections across paired waves, not as a per-seminar analysis.

What is measurement invariance and why does it matter here?

Measurement invariance constrains a segment to mean the same thing at every wave. Without it, an apparent transition might just reflect the segment being defined differently at the end of term. Testing and, where justified, imposing invariance is what lets you interpret movement as genuine change rather than a shifting measurement yardstick.

Can LTA prove that a teaching change caused students to improve?

No. LTA describes movement between segments and can relate that movement to covariates, but on its own it is descriptive. Causal claims need a design that handles confounding and attrition; LTA pairs best with an interrupted-time-series or quasi-experimental framing, and even then differential dropout between waves can bias the transition estimates.

Does aggregate stability mean nothing changed?

Not necessarily. Two waves can show identical means while substantial numbers of students deteriorate and others recover, cancelling out in the average. The value of a transition matrix is precisely that it exposes this compositional churn that a mean-to-mean comparison hides.

Is LTA robust to students dropping out between waves?

Only partly. LTA can incorporate missing-data handling and auxiliary predictors of dropout, but attrition in course evaluation is rarely random — disengaged students are the most likely to disappear from the final wave, which can manufacture or hide transitions. Interpret results cautiously when between-wave attrition is high or selective.

References

Collins, L. M., & Lanza, S. T. (2010). Latent Class and Latent Transition Analysis: With Applications in the Social, Behavioral, and Health Sciences. Wiley. https://doi.org/10.1002/9780470567333
Lanza, S. T., & Collins, L. M. (2008). A new SEM approach to latent transition analysis: Transitions in dating and sexual risk behavior. Developmental Psychology, 44(2), 446-456. https://doi.org/10.1037/0012-1649.44.2.446
Nylund-Gibson, K., Garber, A. C., Carter, D. B., Chan, M., Arch, D. A. N., Simon, O., Whaling, K., Tartt, E., & Lawrie, S. I. (2023). Ten frequently asked questions about latent transition analysis. Psychological Methods, 28(2), 284-300. https://doi.org/10.1037/met0000486

Latent Transition Analysis: Modelling How Student Segments Move Between Evaluation Waves