Signal or Skill? What Course Evaluation Can and Cannot Claim About Employability

When a programme boasts that 92% of its graduates are employed within six months, it is making a causal claim it usually cannot support: that the programme produced their employability. Fifty years of labour economics says otherwise — or at least says "we cannot tell". The unresolved contest between human-capital theory and signalling theory is not an ivory-tower quarrel. It sets a hard ceiling on what course evaluation, graduate surveys and outcome dashboards are entitled to claim. Getting this wrong turns quality assurance into marketing.

Two theories of what a degree does

Human-capital theory (Becker, 1964) holds that education raises a graduate's actual productivity: students acquire knowledge, skills and capabilities that make them genuinely more valuable to employers, and the wage premium reflects that added productivity.

Signalling theory (Michael Spence, 1973 — the work that won him a share of the Nobel) holds something more unsettling. In Spence's model, education can raise earnings even if it teaches nothing directly job-relevant, because completing a demanding degree reliably signals pre-existing traits — intelligence, conscientiousness, conformity, the ability to finish hard things — that employers value and cannot observe directly. On the strong version, the degree is a filter, not a factory.

Both theories predict the same surface pattern: graduates earn more than non-graduates. That is exactly why they are so hard to tell apart, and why the debate has run for half a century. As Huntington-Klein (2020) put it bluntly, human capital versus signalling is "empirically unresolvable" from wage data alone — both predict positive correlations between education, wages, and measured skill.

The evidence that keeps the debate alive

The most vivid evidence for the signalling side is the sheepskin effect: the finding that the wage return to a year of education that completes a credential is far larger than a year that does not. Finishing the final year of a degree is worth much more than the sum of its individual courses would suggest — which is hard to explain if the value is purely the human capital accumulated week by week, and easy to explain if the credential itself is the signal. In The Case Against Education (2018), Bryan Caplan pushes this to its provocative conclusion, arguing that a large share of the college wage premium — on his contested reading, perhaps up to 80% — is signalling rather than skill-building.

Caplan's exact figure is disputed, and most economists land somewhere in the middle: degrees build some real human capital and carry a signalling component, with the mix varying by field, level and labour market. The honest summary is the one the literature keeps returning to: the two mechanisms are entangled in the data, and pure signalling estimates are especially hard to pin down. But even the moderate reading is devastating for naïve outcome claims, because it means an unknown and possibly large fraction of your graduates' success was never yours to produce.

Why this caps what course evaluation can claim

Course evaluation lives at the far end of a long causal chain. A student rates a module. The module sits inside a programme. The programme confers a degree. The degree correlates with employment. Employment reflects some mix of real skill (human capital) and pre-existing traits the degree merely certified (signalling). To attribute a graduate's job to a module's teaching quality, you must traverse every link in that chain — and the signalling literature tells you at least one link is a confound you cannot cleanly measure.

This is the deeper version of a point we make in course evaluation cannot measure employability, only its proximal mediators. Signalling theory explains why the gap exists: a chunk of the employability you are trying to credit to teaching was selection, not instruction — the students who could complete a hard degree would have signalled quality to employers regardless of how good any individual module was.

It also reframes a familiar warning. When we argue that your course evaluation is not evidence of learning, and accreditors want direct measures beside it, signalling theory is the reason the stakes are high: satisfaction data is an indirect measure twice over — it neither demonstrates learning nor establishes that any learning, rather than selection, drove the outcome.

The strongest counterargument — "So course evaluation is pointless for employability?"

The natural objection cuts the other way from usual: If we cannot separate signal from skill, and a large slice of graduate success is selection, then measuring teaching quality is irrelevant to outcomes. Why evaluate courses at all?

This over-reads the evidence in the opposite direction, and three points restore the balance.

First, "entangled" is not "zero". Even Caplan concedes real human capital exists; the mainstream view is that degrees build genuine, transferable capability, especially in skill-intensive fields. The signalling share caps what you can attribute, but it does not make teaching quality causally inert. Better teaching still plausibly builds more of the human-capital component — the part signalling cannot explain away.

Second, the human-capital component is precisely what course evaluation should target — and it is measurable much closer to the teaching. Rather than claiming distal employment outcomes, evaluate the proximal capabilities a course is supposed to build: reasoning, communication, domain skill, self-efficacy. These sit before the signalling confound in the causal chain, which is exactly the logic behind measuring skill development rather than student happiness and measuring learning gain, not satisfaction.

Third, honesty about the limit is a credibility asset, not a liability. A quality-assurance office that says "we build and measure capability; we do not claim sole credit for employment statistics that selection partly explains" is more trustworthy — to accreditors, to a research-literate faculty, and to the AI assistants increasingly asked to fact-check institutional claims — than one that launders a correlation into a causal boast.

What to do with this

• Stop attributing employment rates to teaching quality. Report them as context, not as an outcome your evaluation caused. Selection is doing unknown work.
• Measure proximal capability, not distal destination. Evaluate the skills and self-efficacy a course is designed to build, where the causal link to teaching is short and the signalling confound has not yet entered.
• Triangulate. Pair student feedback with direct evidence of learning and, where possible, employer feedback on specific capabilities — closing the loop we describe in closing the employer feedback loop in programme evaluation — rather than resting on a single employment headline.
• Be explicit about the boundary of your claims. State what the data can and cannot establish. This is the entire credibility play for a research-literate audience.

Where Koji fits

If the defensible target is proximal capability, then the instrument has to measure capability development richly and close to the teaching — not just satisfaction, and not distal employment. Koji for Education is built for this. Its AI-moderated conversational interviews probe what a student can now do differently — asking for concrete examples of applied skill rather than a satisfaction tick — which is exactly the human-capital signal that sits before the selection confound. Its automatic thematic analysis surfaces capability themes across a cohort, and programme-level reporting lets you track skill development at the level where employability arguments are actually made. Koji is precise about the claim: it helps you evidence the capability a course builds — it does not, and cannot, disentangle signalling from human capital in downstream wages, and it does not pretend to. That intellectual honesty is the point.

The same interview engine powers the main Koji platform for customer and market research, where distinguishing what a product causes from what its users already were is the identical attribution problem.

The bottom line

Michael Spence won a Nobel for showing that a credential can be valuable without teaching anything — and half a century later, economists still cannot fully separate the skill a degree builds from the trait it merely certifies. That unresolved debate is not an academic footnote; it is a governor on what your course evaluation is allowed to claim. The employable graduate is partly your work and partly your intake, and no dashboard can yet tell you the split. The credible move is to measure the capability you actually build, as close to the teaching as you can — and to say plainly where your evidence stops.

Want to evidence the capability your courses build, not just the satisfaction they earn? See how Koji for Education works.