Your Comment Categories Sum to 100%: Compositional Data Analysis for Course-Evaluation Shares

In brief

When a course-evaluation number is a share of a fixed whole — the percentage of comments about content versus delivery versus assessment, the split of a fixed points-budget question, or the proportion of class time students say went to each activity — it lives on the simplex, not the real line, and ordinary means, correlations and regressions give misleading answers. Compositional data analysis (CoDA), introduced by John Aitchison in 1982, fixes this by analysing log-ratios between parts rather than the parts themselves. The practical payoff: you stop reporting spurious negative correlations that are pure artefacts of "everything must add to 100%", and your conclusions no longer flip when you drop or merge a category.

What the research says

The foundational result is Aitchison (1982), "The Statistical Analysis of Compositional Data" (JRSS-B). Aitchison showed that a vector of proportions constrained to sum to a constant carries only relative information — the parts are not free to vary independently, so the sample space is the (D-1)-dimensional simplex, not ordinary Euclidean space. Two pathologies follow. First, the negative-bias problem: because the shares must sum to a constant, an increase in one part forces a decrease in the others, so the raw correlation between any two parts is pushed negative regardless of any real relationship — Karl Pearson had already warned about this "spurious correlation" in 1897. Second, subcompositional incoherence: statistics computed on a subset of the parts (after dropping some and renormalising) need not agree with the same statistics computed on the full composition, so your answer depends on which categories you happened to include.

Aitchison's solution was to map the composition off the simplex with log-ratio transformations and do standard statistics in the transformed space. The additive log-ratio (alr) and centred log-ratio (clr) came first; Egozcue, Pawlowsky-Glahn, Mateu-Figueras and Barcelo-Vidal (2003) added the isometric log-ratio (ilr) transformation, which maps the composition to an orthonormal coordinate system so that distances and angles behave normally, curing the singular covariance matrix that clr leaves behind. Pawlowsky-Glahn and Egozcue (2006) give a readable introduction to this "Aitchison geometry" of the simplex; Filzmoser, Hron and Templ (2018) provide a full applied treatment with software. The methodology matured in geology and geochemistry — where a rock sample is literally percentages of minerals — but its logic applies to any constant-sum data: budget shares, time-use diaries, microbiome relative abundances, and, directly, any course-evaluation quantity expressed as a share of a fixed total.

Why it matters for course evaluation in practice

Compositional quantities are more common in evaluation reporting than people notice:

• Theme shares from open text. After coding comments into categories (content, delivery, assessment, workload, support), teams routinely report "42% of comments were about assessment." Those percentages sum to 100% by construction. Comparing them across courses, or correlating "share about assessment" with the overall rating, is a compositional problem — the raw correlation is biased negative because a bigger assessment slice mechanically shrinks every other slice.
• Fixed-budget and constant-sum items. "Distribute 100 points across these five aspects by importance" or "what proportion of class time was lecture / discussion / group work / problem-solving?" produce a composition per respondent.
• Response-option distributions. The split of answers across a Likert scale (% strongly-disagree ... % strongly-agree) is itself a composition, though there the ordinal structure usually makes ordinal regression the better tool.

For genuine shares, CoDA changes three things. (1) Averaging: the correct centre of a set of compositions is the closed geometric mean (Aitchison mean), not the arithmetic mean of each part, which is pulled around by the constraint and need not even sum cleanly to 100%. (2) Association: replace ordinary correlations between parts with log-ratio variances and the variation array, so you measure real co-movement rather than the closure artefact. (3) Modelling: to explain the overall rating from the comment-mix, regress the rating on ilr coordinates of the composition, not on the raw shares — this sidesteps the perfect collinearity that shares carry, since one part is always a linear function of the others.

Limitations and honest caveats

CoDA is not free of trouble. Zeros are the central headache: log-ratios are undefined when a part is zero, and evaluation compositions are full of structural and sampling zeros (a small class with no comments in a category). Rounded and count zeros need multiplicative or Bayesian replacement; essential zeros — a category that genuinely cannot occur — break the model and require a two-part or hurdle formulation. Second, interpretability: ilr coordinates are balances between groups of parts, not the parts themselves, so results must be translated back into plain language for a committee, a real communication cost. Third, the composition must actually be the object of interest: if you care about the absolute number of assessment complaints rather than its share, model the counts directly with count and zero-inflated models, not a composition. Fourth, small classes give noisy shares regardless of method, and no transformation manufactures precision the sample size does not support. Finally, the apparatus assumes the parts are commensurable and the total is a meaningful closure; if a comment can be double-coded into two categories, the "sum to 100%" premise fails and CoDA is the wrong frame.

How Koji incorporates this

Koji is designed to keep compositional and absolute quantities distinct rather than blur them. When Koji's automatic thematic analysis codes open-text answers, it reports both the count of comments per theme and the share, and its comparison views treat shares as relative quantities — differences are read as log-ratios between themes rather than as independent percentage-point moves, so a report does not claim "assessment concern fell 8 points" when the real story is that praise for content rose and pushed every other slice down. For constant-sum and points-allocation questions, Koji's structured question types (ranking, multiple_choice with an allocation, single_choice) capture the composition per respondent, so the geometric-mean centre and log-ratio spread can be computed instead of a naive average of shares. Bias-aware reporting flags the closure trap explicitly: when a dashboard correlates one theme-share against the overall score, Koji surfaces a warning that constant-sum data biases that correlation negative. And because Koji collects data continuously through AI-moderated conversational interviews rather than a single fixed form, it can probe why a category's share moved — asking a follow-up when assessment comments spike — so the compositional shift is explained, not just detected. These mechanisms are designed to mitigate the closure and subcompositional-incoherence problems Aitchison identified; they do not eliminate the zero problem, which still requires a modelling choice on any platform. Teams running mixed product and course research can apply the same log-ratio discipline in Koji's core research platform at koji.so, where budget-split and preference-allocation questions raise identical compositional issues. Pairing the theme-mix with a multiple correspondence analysis of the categorical responses gives a fuller picture of how the parts move together.

Frequently asked questions

What is compositional data in a course-evaluation context?

Any set of numbers that are parts of a fixed whole and sum to a constant — theme shares that total 100%, a points-budget question, or a time-use split. Because the parts cannot vary independently, standard means and correlations are biased, which is exactly what compositional data analysis corrects.

Why is correlating two comment-category shares misleading?

When shares must sum to 100%, one share rising forces the others to fall, so any two parts are pushed toward a negative correlation regardless of any real relationship. This "spurious correlation" is a closure artefact, not a finding. Log-ratio methods remove it.

What is a log-ratio transformation and why use it?

It replaces the raw parts with logarithms of ratios between parts (for example the isometric log-ratio, ilr). This moves the data off the constrained simplex into ordinary space where means, distances and regression behave normally, so standard tools become valid again.

What do I do about categories with zero comments?

Zeros are the main practical obstacle because log-ratios need positive values. Rounded or count zeros are usually handled with a small multiplicative or Bayesian replacement; a category that structurally cannot occur needs a two-part or hurdle model instead. Always report which strategy you used.

When should I NOT use compositional analysis?

When you care about the absolute count rather than the share (model counts directly), when categories overlap so a comment can be double-coded, or when the total is not a meaningful whole. CoDA only applies when "parts of a fixed total" genuinely describes your data.

Does averaging percentages across courses need special care?

Yes. The arithmetic mean of shares is distorted by the constraint and can even fail to sum to 100%. Use the closed geometric mean (the Aitchison mean), which respects the simplex geometry and gives a coherent central composition.

References

• Aitchison, J. (1982). The Statistical Analysis of Compositional Data. Journal of the Royal Statistical Society: Series B (Methodological), 44(2), 139-160. https://doi.org/10.1111/j.2517-6161.1982.tb01195.x
• Aitchison, J. (1986). The Statistical Analysis of Compositional Data. London: Chapman & Hall.
• Egozcue, J. J., Pawlowsky-Glahn, V., Mateu-Figueras, G., & Barcelo-Vidal, C. (2003). Isometric Logratio Transformations for Compositional Data Analysis. Mathematical Geology, 35(3), 279-300. https://doi.org/10.1023/A:1023818214614
• Pawlowsky-Glahn, V., & Egozcue, J. J. (2006). Compositional data and their analysis: an introduction. Geological Society, London, Special Publications, 264(1), 1-10. https://doi.org/10.1144/GSL.SP.2006.264.01.01
• Filzmoser, P., Hron, K., & Templ, M. (2018). Applied Compositional Data Analysis: With Worked Examples in R. Cham: Springer. https://doi.org/10.1007/978-3-319-96422-5

Related resources

• Mapping the Patterns You Cannot Average: Multiple Correspondence Analysis for Categorical Course-Evaluation Data
• Your Outcome Is a Proportion, Not a Mean: Beta Regression for Course Evaluation
• Count Models for Course Evaluation: Why You Should Not Average the Number of Comments
• Topic Modeling for Open-Text Course Evaluations
• Ordinal Regression for Course-Evaluation Data