Your Class Doesn't Have an Average Opinion. It Has Three or Four Distinct Ones.
Averaging a course evaluation assumes every student experienced roughly the same course. Q-methodology, a technique for the systematic study of subjectivity dating to 1935, does the opposite: it identifies the small number of distinct, shared viewpoints inside a cohort. Here is what it reveals that a mean erases — and where its limits are.
Koji Education Team
Product · August 9, 2026
Bottom line: A mean rating treats a course as one experience with some noise around it. Often that is false: a single seminar is experienced as three or four genuinely different courses by three or four groups of students, and the "4.1" belongs to none of them. Q-methodology — a technique for the systematic study of subjectivity, introduced by William Stephenson in 1935 — is built to surface exactly those distinct shared viewpoints. It will not tell you how common each viewpoint is, but it will tell you, rigorously, what they are — which is precisely the thing averaging destroys.
What Q-methodology actually is
Q-methodology first appeared in a 1935 letter to Nature from William Stephenson, then at the University of London, as a deliberate inversion of conventional factor analysis. Where ordinary ("R") factor analysis looks for patterns across variables — correlating survey items over many people — Stephenson's move was to correlate people over many statements. This "by-person" factor analysis is the whole point: it groups together participants who ranked a set of statements in similar ways, and each resulting factor is a distinct, shared point of view. The definitive modern guide is Watts and Stenner's 2012 Doing Q Methodological Research: Theory, Method and Interpretation (Sage), which lays out Stephenson's concepts of subjectivity, concourse theory and abduction; the foundational applied text remains Steven Brown's 1980 Political Subjectivity.
In practice, a Q study works like this. You assemble a broad set of statements about the course — drawn from student comments, module aims, common complaints — called the concourse, and sample it down to a manageable set (often 30-60 statements). Each participant then performs a Q-sort: they rank the statements into a forced, roughly normal distribution, from "most like how I experienced this course" to "least like it". Because the ranking is forced and relational, students cannot simply agree with everything the way they do on a Likert battery; they must trade off, which is where the structure of their viewpoint shows itself. By-person factor analysis of those sorts yields a handful of factors, each a composite viewpoint that real students loaded onto.
What it shows that an average cannot
Consider a research-methods module. The mean says 4.1/5, "good, slightly below the department norm", and nobody learns anything. A Q study of the same cohort might reveal three viewpoints: one group experienced it as rigorous and career-relevant and wanted more statistics; a second found the pace brutal and the support thin and was quietly drowning; a third liked the teacher but saw no point to the content. Those are not three points on one scale — they are three different courses. The average is a weighted blur of all three, and any action taken on it (tweak the mean upward) will help none of them.
This is a different operation from the ones the corpus already covers. It is not the ordinal-mean problem addressed in why averaging Likert scores misleads or the interval-scale fix in Rasch measurement; those improve the number. It is not thematic analysis of open text, which codes what individuals say. Q-methodology maps the structure of shared subjectivity — how many distinct ways of making sense of the course exist, and what each one prioritises and rejects. It sits between the qualitative and the quantitative: rigorous and factor-analytic, but aimed at holistic viewpoints rather than variable means.
The limitation you must state first
Q-methodology has a hard boundary, and pretending otherwise discredits it: it tells you what viewpoints exist, not how prevalent they are. A Q study deliberately uses a small, purposive participant set (the P-set), because its unit of analysis is the viewpoint, not the population. Finding three factors does not mean a third of students hold each; the proportions in a Q study are an artefact of who you recruited, not an estimate of the cohort. Anyone who reports "42% of students hold viewpoint B" from a Q study has misunderstood the method.
So Q-methodology does not replace a representative survey — it answers a different question. The mature design pairs them: use Q to discover the distinct viewpoints in a cohort, then, if you need prevalence, write a short survey that lets the whole class self-identify with the viewpoints Q found. Q gives you the map; the survey counts the territory.
But isn't this too heavy for routine evaluation?
A fair objection. A full Q study — building a concourse, running physical card sorts, extracting and rotating factors — is a research project, not a Tuesday-afternoon module survey, and it is overkill for a course that is working fine. Two things soften the objection. First, you do not Q-sort every module; you reach for it when a mean is stubbornly mid-range or bimodal and you genuinely do not understand why students disagree — the situations where an average is most actively misleading. Second, the underlying insight — stop averaging away difference, start identifying distinct shared experiences — can be built into routine tools even when a formal Q-sort is impractical. The value is the mindset as much as the technique: a cohort is a set of viewpoints, not a distribution around a mean.
Where Koji fits
Koji operationalises the Q insight without requiring every programme to run a textbook Q study. Its AI-moderated conversational interviews are designed to surface why a student experienced the course as they did and to follow that thread — the raw material of a viewpoint, not a single rating — and its automatic thematic analysis clusters those responses into the distinct, recurring patterns of experience within a cohort rather than flattening them into one number. The ranking question type lets you put a Q-style forced trade-off directly to students — rank these aspects of the course from most to least valuable — which yields the relational, structured data a mean cannot. And because reporting separates and names the patterns instead of collapsing them, a programme lead sees the three-different-courses reality rather than the blurred 4.1. Teams running wider audience or user research can apply the same viewpoint-first engine on the main Koji platform.
To be precise: Koji does not run formal by-person factor analysis, and it does not claim to. It mitigates the averaging problem by capturing and clustering the distinct experiences Q-methodology is designed to reveal, so that action can be targeted at the group that actually needs it rather than at a statistical fiction.
Frequently asked questions
What is Q-methodology in one sentence? It is a research technique, introduced by William Stephenson in 1935, that uses by-person factor analysis of ranked statements (Q-sorts) to identify the distinct, shared viewpoints held within a group.
How is Q-methodology different from a normal survey? A normal (R-methodology) survey correlates items across many people to find patterns among variables and estimate prevalence. Q correlates people across many statements to find distinct viewpoints. Q tells you what viewpoints exist; it does not tell you how common each one is.
Can Q-methodology tell me what percentage of students hold each view? No. Q uses a small, purposive participant set and its proportions are an artefact of recruitment, not a population estimate. To measure prevalence, follow a Q study with a representative survey that lets students self-identify with the viewpoints Q found.
When should a programme use Q rather than a standard evaluation? When a mean is stubbornly mid-range or clearly bimodal and you do not understand why students disagree. Q is diagnostic — it explains the disagreement an average hides — rather than a routine monitoring tool for every module.
Does Koji do Q-methodology? Not in the formal factor-analytic sense. Koji applies the core Q insight — identify distinct shared viewpoints instead of averaging them away — through conversational interviews, a ranking question type, and thematic clustering of responses.
Tired of averages that describe no actual student? See how Koji for Education surfaces the distinct viewpoints inside your cohort.