SALG: Can Students Reliably Report Their Own Learning Gains?

The short answer

The Student Assessment of their Learning Gains (SALG) instrument reframes the course questionnaire around a better question: instead of asking whether students were satisfied, it asks how much each element of the course helped their learning and what gains they made in specific skills, concepts and attitudes. That shift — from liking to learning — is genuinely valuable and is what makes SALG worth knowing. But the honest verdict from the wider literature is that self-reported learning gains are a perception of learning, not a measurement of it: they correlate weakly with objective gains and are vulnerable to recall and estimation error. Use gains-framed questions to understand the student experience of learning; do not treat the numbers as a proxy for actual achievement.

Bottom line: SALG''s design innovation — anchoring items on gains and on which course elements helped, rather than on satisfaction — produces more actionable feedback than a standard rating form. But self-reported gains are not a substitute for direct measures of learning (tests, concept inventories, capstone performance). Triangulate them; do not report them alone as evidence of learning.

What the research says

SALG grew out of NSF-funded reform of introductory college chemistry in the United States. Developed by Elaine Seymour, Stephen Carroll and Tim Weston, version 1 was released in 1997 for an introductory chemistry context, and version 3 (2007) was generalised to any discipline and pedagogy. The instrument''s structure is its defining feature. Rather than a satisfaction battery, SALG asks students to rate, on a "help/gains" scale (roughly no help to great help, no gains to great gains), two distinct things: (1) how much each aspect of the class — the labs, the readings, the assessment, the group work — helped their learning, and (2) what gains they made in understanding key concepts, in specific skills, in attitudes, and in integrating information. Instructors customise the items to their own course goals.

The development evidence is substantial. According to the instrument''s documentation (PhysPort awards it a "Silver" validation), SALG was built on over 300 student interviews, in which researchers found that students were effective at self-reporting what they had gained from particular aspects of a course — but not at reporting what they merely liked. It was field-tested across roughly 14 courses at 8 institutions and later broadened to some 30 faculty across disciplines; it has since been used by tens of thousands of instructors. A later psychometric study of an adapted SALG following instruction in stereochemistry (Chemistry Education Research and Practice, 2016) examined its latent structure and reported reasonable reliability for the perceived-gains constructs, while noting — as most SALG researchers do — that adapted versions need their own validation.

That last point is the pivot to the critical literature. The most rigorous challenge to any self-reported-gains instrument is Porter (2013), Self-reported learning gains: A theory and test of college student survey response (Research in Higher Education, 54(2), 201–226). Porter argued that answering a learning-gains item requires a cognitive feat students cannot reliably perform: recalling their prior state, assessing their current state, and computing the difference — all for an abstract construct. His analysis found self-reported gains had little relationship to objective measures of learning and behaved more like a general attitude than a measurement. Bowman (2010), studying first-year students (American Educational Research Journal, 47(2), 466–496), reached a compatible conclusion: self-reported gains correlated only weakly with longitudinal, objectively measured gains, and the gap was systematic, not random. The DeLauriers et al. active-learning experiments add a further twist: students can feel they learned less in the classes where they objectively learned more, so perceived gains and real gains can even point in opposite directions.

The synthesis is nuanced. SALG''s framing is a real improvement over satisfaction surveys because it directs attention to learning and to actionable course elements. But the construct it measures is perceived learning, and perceived learning is a fallible, sometimes inverted, indicator of the real thing.

Why it matters for course evaluation in practice

For a quality-assurance office, SALG offers two concrete advantages and one clear boundary.

The first advantage is diagnostic specificity. A satisfaction mean tells you a course scored 4.1; a gains-framed instrument tells you students felt the problem sets drove most of their learning while the lectures added little — an insight a programme director can act on. Because SALG ties items to course elements, the feedback points at design levers rather than at the instructor''s personality.

The second is construct hygiene. The confounds literature shows satisfaction ratings absorb grading leniency, charisma, and course difficulty. Asking about gains does not remove those confounds, but it re-orients the whole exercise toward the outcome the institution actually cares about — learning — and away from consumer satisfaction, which the student-as-consumer literature warns can be actively counterproductive.

The boundary is equally important: gains data is not outcome data. For accreditation evidence of learning outcomes (ESG/ENQA, AACSB assurance-of-learning, ABET), a self-reported-gains score cannot stand in for direct assessment. It belongs in the "indirect measures" column, triangulated with the direct measures — coursework, exams, concept inventories — never as a replacement for them.

Limitations and honest caveats

• Self-report validity is the central problem. Porter (2013) and Bowman (2010) both show self-reported gains relate weakly to objective learning. The polite reading is that SALG measures a meaningful experience of learning; the strict reading is that it does not measure learning at all.
• The estimation task is cognitively unrealistic. Students are asked to compute a before/after difference on an abstract construct they cannot accurately recall. Response-shift bias compounds this: the very act of taking a course changes the yardstick students use to judge their prior knowledge.
• Perceived and actual gains can diverge — even invert. In active-learning settings, fluency and effort cues can make students underrate learning that objectively occurred, and overrate learning from polished but shallow instruction.
• Adapted versions need fresh validation. SALG is designed to be customised, but a validity study of one adapted version does not license the next. Reliability and structure should be re-checked when items change substantially.
• Discipline and origin. SALG was developed and validated primarily in STEM courses in one national system; its behaviour in humanities, in small seminars, and across European languages and grading cultures is under-studied.

These caveats do not condemn SALG. They locate it correctly: a strong formative and indirect instrument, not a measure of achievement.

How Koji incorporates this

Koji takes SALG''s core insight — ask about gains and about which course elements helped, not about satisfaction — and addresses the response-process problem that Porter identified, without pretending to solve it.

• Gains-framed question design. Koji supports scale, single_choice, ranking and open_ended items, so an instrument can be built around "how much did this element help your learning" and "what can you now do that you could not before" rather than around satisfaction — SALG''s framing, natively.
• Conversational probing to reduce estimation error. Porter''s critique is that a bare gains rating is a guess. Koji''s AI-moderated interview can follow a gains item with a concrete probe — "Give an example of something you can do now that you couldn''t at the start" — pushing students from an abstract self-estimate toward specific, evidenced claims that a reviewer can weigh. This is designed to mitigate, not eliminate, the self-report validity gap.
• Separating elements from gains. Because Koji can structure a study around distinct course elements and then thematically analyse the open-text explanations, it reproduces SALG''s element-by-element diagnostic while surfacing why an element helped.
• Triangulation and honest labelling. Koji''s reporting is designed to present perceived-gains data as an indirect measure to be triangulated with direct outcomes, not as a stand-in for achievement — the boundary the accreditation frameworks insist on.
• Formative timing. SALG works best mid-course, when feedback can still change the experience; Koji supports mid-cycle collection and closing-the-loop action tracking so a gains signal turns into a change rather than a post-mortem.

Koji''s core research platform at koji.so applies the same gains-versus-satisfaction discipline to product and customer research, where "how much did this help you accomplish your goal" is a far more useful question than "how satisfied are you."

Related resources

• Why "I Learned a Lot" Can''t Be Compared Across Courses: Reference Bias
• Response-Shift Bias: Why Self-Reported Learning Gains Can Mislead
• Students Rate the Classes Where They Learn the Most Lower: The Feeling-of-Learning Gap
• Engagement Surveys vs Course Evaluations: What NSSE Measures
• Teacher Clarity Predicts Learning Better Than Charisma
• Does the Professor Students Rate Highest Teach Them Most? Value-Added Evidence

References

• Seymour, E., Wiese, D., Hunter, A., & Daffinrud, S. M. (2000). Creating a better mousetrap: On-line student assessment of their learning gains. Paper presented at the National Meeting of the American Chemical Society, San Francisco, CA. (SALG instrument: https://salgsite.net)
• Porter, S. R. (2013). Self-reported learning gains: A theory and test of college student survey response. Research in Higher Education, 54(2), 201–226. https://doi.org/10.1007/s11162-012-9277-0
• Bowman, N. A. (2010). Can 1st-year college students accurately report their learning and development? American Educational Research Journal, 47(2), 466–496. https://doi.org/10.3102/0002831209353595
• Vishnumolakala, V. R., et al. (2016). Latent constructs of the students'' assessment of their learning gains instrument following instruction in stereochemistry. Chemistry Education Research and Practice, 17. https://doi.org/10.1039/C5RP00214A