Evaluating Online and Blended Teaching: The Community of Inquiry Framework

In brief: When you evaluate an online or blended course with the same generic "satisfaction" items you use for a lecture hall, you measure the wrong thing. The Community of Inquiry (CoI) framework (Garrison, Anderson & Archer, 2000) decomposes a meaningful online educational experience into three measurable elements — teaching presence, social presence and cognitive presence — and the validated 34-item CoI survey (Arbaugh et al., 2008) operationalises them with strong internal consistency. For European quality assurance, CoI gives online course evaluation a construct-grounded structure that explains why a course works, not just whether students liked it.

What the research says

The Community of Inquiry framework originated in Garrison, Anderson & Archer (2000), "Critical inquiry in a text-based environment: Computer conferencing in higher education," published in The Internet and Higher Education. The authors argued that a worthwhile educational experience is collaborative and constructivist, and emerges from the interaction of three overlapping "presences":

• Teaching presence — the design, facilitation and direction of cognitive and social processes: setting the curriculum and activities, guiding discussion, and providing direct instruction. This is the element most under an instructor's control and the one most relevant to evaluating teaching.
• Social presence — the degree to which participants feel socially and emotionally connected, able to project themselves as real people and to interact meaningfully. It is the affective substrate that makes collaboration possible at a distance.
• Cognitive presence — the extent to which learners construct and confirm meaning through sustained reflection and discourse, often described through a practical inquiry cycle of triggering event, exploration, integration and resolution.

The key claim is that a deep educational experience sits at the intersection of the three: teaching presence sustains a climate (with social presence) and selects content and challenge (with cognitive presence) that together support higher-order learning.

The framework became measurable with Arbaugh et al. (2008), "Developing a community of inquiry instrument," also in The Internet and Higher Education. Across a multi-institutional sample, they validated a 34-item survey (13 teaching-presence items, 9 cognitive-presence items, 12 social-presence items — administered on a Likert scale). The instrument showed high internal consistency — reported reliabilities of roughly 0.94 for teaching presence, 0.91 for social presence and 0.95 for cognitive presence — and a clean three-factor structure, suggesting a stable measure usable for large-scale, cross-institutional and cross-disciplinary work.

A decade of subsequent research reinforced and complicated the picture. Garrison, Anderson & Archer (2010), in their retrospective "The first decade of the Community of Inquiry framework," report that the model became one of the most widely used lenses for studying online and blended learning, with teaching presence repeatedly emerging as a strong predictor of perceived learning and satisfaction. Independent psychometric work has broadly supported the three-factor instrument while debating whether some sub-dimensions (e.g., within social presence) should be refined.

Why does this matter relative to a generic course-experience questionnaire? Instruments such as the Course Experience Questionnaire (Ramsden, 1991) capture perceived teaching quality well for conventional settings, but they were not designed around the distinctive mechanics of asynchronous, technology-mediated learning — the felt presence of others, the labour of sustaining inquiry without a shared room. CoI fills that gap with constructs specific to the online context.

Why it matters for course evaluation in practice

European higher education has shifted decisively toward online, hybrid and blended provision, and quality assurance has had to follow. The ESG (Standards and Guidelines for Quality Assurance in the European Higher Education Area) expect institutions to monitor and act on the student learning experience; for online provision, a CoI-structured evaluation offers three practical advantages.

1. Diagnostic, not just evaluative. A 4.1/5 satisfaction score is inert. A CoI profile that shows high teaching presence, adequate cognitive presence, but weak social presence tells a programme director exactly where to intervene — for example, by redesigning discussion facilitation and peer interaction rather than rewriting lecture content.

2. Attributes outcomes to design choices. Because teaching presence is decomposed into design, facilitation and direct instruction, evaluation evidence maps onto things the institution can change: the structure of the module, the rhythm of facilitation, the clarity of instructions. This is the difference between feedback that closes the loop and feedback that merely scores.

3. Comparability across modalities. Using CoI for online provision and a parallel, construct-aware instrument for on-campus provision lets QA teams compare like constructs rather than forcing one generic form onto incompatible settings — a recurring problem in modality comparisons.

4. Longitudinal tracking of redesign impact. Because the three presences are stable, separable constructs, an institution can monitor them across successive cohorts and test whether a specific intervention — say, introducing structured peer discussion to lift social presence — actually moved the targeted construct. A generic satisfaction trend line cannot do this; it conflates everything into one number and so cannot tell improvement in facilitation apart from a lucky cohort. CoI gives quality assurance a measurable, attributable signal of whether online-course redesign worked.

In short, CoI turns "did students like the online course?" into "which component of the online learning experience is strong, which is weak, and what should we redesign — and did the redesign work?"

Limitations and honest caveats

CoI is a strong framework, not a finished science, and a critical reader should keep several reservations in view.

• Self-report and perception. The CoI survey measures perceived presence, not directly observed behaviour or learning. High cognitive-presence scores indicate students felt they reached integration and resolution; they are not an objective measure of learning outcomes, and the gap between perceived and actual learning is well documented elsewhere.
• The cognitive-presence/outcomes link is contested. While teaching presence reliably predicts satisfaction and perceived learning, the empirical link between measured cognitive presence and objective achievement is more variable across studies. Treat cognitive-presence scores as a process indicator, not a grade predictor.
• Construct overlap. The three presences are conceptually distinct but empirically correlated; some psychometric studies report cross-loadings, and debates continue about sub-dimension structure (especially social presence). The clean three-factor result is robust but not unanimous.
• Context of origin. The framework grew out of text-based computer conferencing. Modern courses use video, synchronous sessions, AI tutors and large-enrolment formats, so the operationalisation needs thoughtful adaptation rather than literal transplantation.
• Generalizability. Much CoI validation comes from North American and graduate/online-specialist samples; cross-cultural and discipline-specific measurement invariance should not be assumed for a European undergraduate context without checking.

These caveats argue for using CoI as a structured lens alongside other evidence (outcomes, completion, qualitative depth), not as a single verdict.

How Koji incorporates this

Koji is designed to collect course-evaluation evidence that has structure and explains itself — which is exactly what the CoI framework demands. The mapping is concrete.

• Construct-aligned question sets. A QA team can build a Koji evaluation whose items map to teaching, social and cognitive presence using mixed scale, single_choice and open_ended types — capturing a CoI-style profile rather than one undifferentiated satisfaction number.
• AI-moderated interviews that test presence in depth. Where a static survey asks "I felt part of a learning community" on a 1-to-5 scale, Koji's conversational interviewer can follow up — when did you feel connected, and when isolated? what did the instructor do that helped you make sense of the material? This probes the facilitation and social-presence mechanisms that drive the scores, turning a rating into a redesign brief.
• Automatic thematic analysis organises open text into themes that naturally align with the three presences (e.g., "unclear instructions" maps to teaching presence; "no peer interaction" to social presence), so the qualitative evidence is interpretable at programme level.
• Mid-cycle / formative collection lets teaching presence be diagnosed and adjusted during an online module, when facilitation can still change the experience — not only in a post-mortem.
• Bias-aware, dispersion-sensitive reporting presents per-construct distributions and theme prevalence, supporting the framework's intended diagnostic use rather than a single headline mean.

As always, Koji frames this as designed to operationalise the CoI constructs more richly than a static form — not as a claim that conversational data perfectly captures cognitive presence. Koji's core platform at koji.so runs the same engine for product and customer research, where understanding why an experience works is the whole point.

Related resources

• Online vs Face-to-Face Modality and Student Evaluations
• The Course Experience Questionnaire (CEQ) and Perceived Teaching Quality
• What Do Student Evaluations Measure? Marsh on Multidimensionality
• SERVQUAL and HEdPERF Service-Quality Gap Models in Course Evaluation
• Student Engagement Surveys (NSSE) vs Course Evaluations
• Online vs Paper Course Evaluation Response Rates

References

• Garrison, D. R., Anderson, T., & Archer, W. (2000). Critical inquiry in a text-based environment: Computer conferencing in higher education. The Internet and Higher Education, 2(2–3), 87–105. https://doi.org/10.1016/S1096-7516(00)00016-6
• Arbaugh, J. B., Cleveland-Innes, M., Diaz, S. R., Garrison, D. R., Ice, P., Richardson, J. C., & Swan, K. P. (2008). Developing a community of inquiry instrument: Testing a measure of the Community of Inquiry framework using a multi-institutional sample. The Internet and Higher Education, 11(3–4), 133–136. https://doi.org/10.1016/j.iheduc.2008.06.003
• Garrison, D. R., Anderson, T., & Archer, W. (2010). The first decade of the community of inquiry framework: A retrospective. The Internet and Higher Education, 13(1–2), 5–9. https://doi.org/10.1016/j.iheduc.2009.10.003
• Ramsden, P. (1991). A performance indicator of teaching quality in higher education: The Course Experience Questionnaire. Studies in Higher Education, 16(2), 129–150. https://doi.org/10.1080/03075079112331382944