The Nominal Group Technique: Structured Group Feedback That Ranks What Students Actually Care About

In short

The Nominal Group Technique (NGT) is a structured, facilitated method that turns a group of students into a ranked, consensus list of what matters most about a course — combining the breadth of a survey with the depth of a focus group while neutralising the loud-voice problem of both. Developed by Delbecq and Van de Ven and applied to higher-education evaluation by Chapple and Murphy (1996) and others, NGT has students silently generate ideas, share them round-robin, discuss them, and then privately rank them. The output is not an average rating but a prioritised action list with built-in agreement about relative importance — exactly the artefact a programme team needs and a Likert survey cannot produce.

---

What the research says

The Nominal Group Technique was introduced by André Delbecq and Andrew Van de Ven and codified in their 1975 book Group Techniques for Program Planning. It is "nominal" because, for key stages, the group is a group in name only: members work in silence, so the process captures independent judgement before any social influence sets in. A canonical NGT session with 6–10 participants and a facilitator runs in four stages:

1. Silent idea generation. In response to a single, carefully worded question (e.g. "What helped or hindered your learning on this module?"), each student writes ideas privately for several minutes.
2. Round-robin recording. The facilitator collects one idea per person per turn, in rotation, until all ideas are exhausted — every contribution is recorded with equal standing and no attribution.
3. Clarification and discussion. The group discusses each item only to clarify meaning and merge duplicates, not to argue or persuade.
4. Private ranking/voting. Each student privately rank-orders or scores the top items. The facilitator aggregates the votes into a prioritised list with a visible measure of consensus.

The application of NGT to course and curriculum evaluation is well documented. Chapple and Murphy (1996), in Assessment & Evaluation in Higher Education, argue that NGT "extends the evaluation of students' teaching and learning experiences" beyond what questionnaires capture, because it surfaces students' own categories of what matters rather than imposing the designer's item list, and because the ranking stage produces an actionable priority order. Dobbie, Rhodes, Tysinger and Freeman (2004), in Family Medicine, report using a modified NGT as a practical curriculum-evaluation tool that complements forms, surveys and focus groups, and found it efficient at generating consensus quickly. Earlier work such as O'Neil and Jackson (1983) in Studies in Higher Education used NGT to initiate curriculum development, and the technique has been adopted across nursing, medical and general higher education because it produces immediate, visible results in a single session.

Two features distinguish NGT from the qualitative methods already common in evaluation. First, unlike an ordinary focus group, the silent-generation and private-voting stages prevent a few confident students from anchoring the discussion — the evidence base repeatedly cites reduced dominance and more equal participation as NGT's signature advantage. Second, unlike open-text survey comments, NGT delivers prioritisation: you learn not just that ten things were mentioned, but which three the cohort collectively judges most important.

Why it matters for course evaluation in practice

Most course-evaluation instruments answer the question "how did students rate the course?" Very few answer "what would students fix first?" NGT is built to answer the second question, and that makes it valuable at specific points in the quality cycle:

• Diagnosing a struggling module. When a module scores poorly on a survey, the numbers rarely say why. An NGT session with a volunteer cohort produces a ranked list of concrete causes and remedies in under an hour — a far faster route to an improvement plan than another round of surveys.
• Closing the feedback loop credibly. Because students collectively rank the priorities, the programme team can act on the top item and report back "you told us X was your number-one issue; here is what we changed." That visible responsiveness addresses the well-evidenced problem that students doubt anyone acts on their feedback.
• Co-design and partnership. NGT operationalises the students-as-partners principle: students help define what counts as a problem, not merely rate pre-set items.
• Triangulation. NGT complements quantitative course evaluation and other qualitative methods such as the critical incident technique and most significant change, giving a programme team convergent evidence from independent methods.

For accreditation, an NGT record is strong evidence of student engagement in quality enhancement — it demonstrates a structured, documented mechanism for gathering and acting on the student voice, which frameworks such as the ESG expect.

Limitations and honest caveats

NGT is a specialist tool, not a replacement for routine evaluation, and a critical reader should note its real constraints:

• It does not scale like a survey. A session handles 6–10 students well; large cohorts require multiple parallel sessions and careful aggregation, and the students who attend are self-selected volunteers, so the sample is not representative in the way a census-style survey aspires to be.
• Facilitator effects are real. The neutrality and skill of the facilitator materially shape the output. A leading facilitator can bias idea generation or the framing of items during clarification. Reliability across facilitators is a genuine methodological concern.
• The single question matters enormously. NGT's entire yield depends on one well-designed prompt. A vague or double-barrelled question produces a muddled list — the same question-design discipline that governs surveys applies here.
• Ranking consensus can mask disagreement. An aggregated priority order can hide a genuinely split cohort. Reporting the spread of votes, not just the mean rank, is essential — and even then, NGT captures the priorities of those in the room on that day.
• Evidence base is largely descriptive. Much of the higher-education NGT literature reports successful applications rather than controlled comparisons against other methods; claims that NGT is superior should be made cautiously.

None of these undermines NGT as a complement; they simply mean it should be positioned as a focused, facilitated method used alongside — not instead of — broad-based evaluation.

How Koji incorporates this

Koji for Education is a scaled, asynchronous evaluation platform rather than an in-person workshop, but the design logic of NGT — independent generation, equal voice, explicit prioritisation — is built into how Koji collects and analyses feedback:

• AI-moderated conversational generation mirrors silent idea generation. Each student's Koji interview is conducted privately with an AI moderator that probes their own reasoning, so ideas are generated independently and free from peer influence — the same anti-dominance property that makes NGT's silent stage valuable, achieved at cohort scale rather than for a room of ten volunteers.
• Ranking as a first-class question type. Koji supports a native ranking item (alongside open_ended, scale, single_choice, multiple_choice and yes_no), so students can prioritise options directly — reproducing NGT's private-voting stage across the whole cohort instead of a self-selected group.
• Automatic thematic analysis reproduces round-robin consolidation. Where an NGT facilitator manually merges duplicate ideas, Koji's automatic thematic analysis of open-text responses clusters semantically similar points into themes and quantifies how many students raised each — turning free-text into a prioritised, de-duplicated list, which is precisely the artefact NGT hand-produces.
• Consensus made visible, not averaged away. Koji's reporting is designed to show the distribution and prevalence of each theme, so a genuinely split cohort is not hidden behind a single ranked mean — addressing the "consensus can mask disagreement" caveat above.
• Closing-the-loop action tracking. Koji can attach follow-up actions to the top-ranked themes and track whether they were addressed, operationalising the credibility benefit that makes NGT persuasive to students.

Koji is designed to mitigate the loud-voice and low-actionability problems that NGT was invented to solve — not to claim it reproduces a facilitated workshop perfectly. For teams running the same prioritisation logic on customers or users rather than students, Koji's core research platform at koji.so applies the identical AI-moderated interview and ranking engine to product and market research.

Related resources

• The Most Significant Change Technique: Story-Based Course Evaluation
• The Critical Incident Technique for Course Feedback
• Students as Partners, Not Just Respondents: Co-Designing Course Evaluation
• Q-Methodology: Surfacing the Distinct Viewpoints Students Hold About a Course
• How to Design Open-Ended Course Evaluation Questions That Get Useful Answers

References

• Delbecq, A. L., Van de Ven, A. H., & Gustafson, D. H. (1975). Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Scott, Foresman.
• Chapple, M., & Murphy, R. (1996). The Nominal Group Technique: extending the evaluation of students' teaching and learning experiences. Assessment & Evaluation in Higher Education, 21(2), 147–160. https://doi.org/10.1080/0260293960210204
• Dobbie, A., Rhodes, M., Tysinger, J. W., & Freeman, J. (2004). Using a modified nominal group technique as a curriculum evaluation tool. Family Medicine, 36(6), 402–406. https://pubmed.ncbi.nlm.nih.gov/15181553/
• O'Neil, M. J., & Jackson, L. (1983). Nominal Group Technique: a process for initiating curriculum development in higher education. Studies in Higher Education, 8(2), 129–138. https://doi.org/10.1080/03075078312331379014