Employers Stopped Asking for the Degree. What Should Programme Evaluation Measure Now?

The short answer: A slow, uneven shift is under way in how graduates are hired — from screening on the credential toward screening on demonstrable skills. If that shift is real, then programme evaluation that asks only "were you satisfied with your degree experience?" is measuring the wrong construct: it captures satisfaction with a signal that the labour market increasingly discounts, while saying nothing about whether the programme built the transferable capabilities employers now test for. But the story is easy to oversell — the gap between skills-based-hiring rhetoric and practice is enormous — and course evaluation cannot measure employability directly in any case. The defensible move is not to bolt an "employability score" onto your survey. It is to evaluate the proximal skill development a programme can genuinely influence, and to triangulate it with employer and graduate evidence.

What is actually changing in the labour market

Start with the evidence, because the topic is drowning in vendor optimism. The World Economic Forum's Future of Jobs Report 2025 finds that employers expect 39% of workers' core skills to change by 2030 — a large disruption, though notably down from the 44% forecast in 2023, which is itself a useful signal that the "everything is changing" narrative overshoots. The same report finds 63% of employers identify skills gaps as the single biggest barrier to business transformation, and that 59% of the workforce will need reskilling or upskilling by 2030, with 11% likely to miss out. Analytical thinking remains the most-valued core skill, cited as essential by roughly seven in ten employers.

On the demand side in Europe, the pressure is concrete. ManpowerGroup's 2025 Talent Shortage survey reports around 74% of European employers struggling to fill roles with the skilled talent they need, and Cedefop estimates the existing skills of the EU workforce fall roughly one-fifth short of what jobs require for full productivity. Employers are, in aggregate, telling us that the skills a graduate can demonstrate matter more than ever relative to the box that says "degree, subject, class".

The National Association of Colleges and Employers' Job Outlook 2026 reports that around 70% of employers now use skills-based hiring for entry-level roles, up from 65% the year before. The direction is real. The magnitude is where honesty is required.

The counterargument, up front: skills-based hiring is mostly talk

The single most important corrective to this topic comes from Harvard Business School and the Burning Glass Institute. Joseph Fuller's 2025 analysis of employers that publicly dropped degree requirements found that fewer than 1 in 700 new hires actually benefited from the reforms. Only about 37% of the companies studied made any real change to whom they hired; for the rest, "skills-first" was a press release. And the share of job postings that require a degree has, in some analyses, ticked upward again since 2024.

So the correct reading is not "the degree is dead." It is: a genuine but slow structural shift is happening, overlaid with a great deal of rhetoric that has not yet changed behaviour. A programme that panics and reorients entirely around a labour-market transformation that is 90% announcement would be making a serious mistake. This is exactly the kind of over-rotation our piece on what course evaluation can and cannot claim about employability warns against.

Which sets up the deeper problem.

Why "measure employability" is the wrong instruction

Even granting the shift, you cannot ask your course evaluation to measure employment outcomes. Whether a graduate gets hired depends on the labour market, geography, the economic cycle, personal circumstances, and luck — the overwhelming majority of which a single module or even a whole programme does not control. Attributing a graduate's job to a specific course is the signalling-versus-human-capital problem in miniature: the credential may predict the outcome without producing it. An "employability" item on a Likert form measures a student's optimism, not a causal effect.

What a programme can legitimately measure is proximal: the skill development it plausibly contributed to, at the level where the causal claim is defensible. Did students report — and can they demonstrate — improvement in analytical reasoning, in communicating a technical argument, in collaborating on an ambiguous problem? Those are the capabilities employers say they screen for, they are the mediators between a course and a career, and they are close enough to the teaching that a programme can honestly take partial credit or partial responsibility. This is the shift from measuring satisfaction to measuring skill development that the graduate-outcomes agenda actually needs.

What a skills-aware evaluation looks like

If the goal is to evaluate proximal skill development rather than satisfaction with a credential, three design changes follow:

1. Anchor to a real skills taxonomy, not ad-hoc adjectives. Europe already has shared vocabularies — ESCO for skills and occupations, and the competence frameworks (DigComp, EntreComp, GreenComp) — that let a programme map its intended outcomes to language employers and accreditors recognise. Mapping evaluation questions onto these frameworks turns "did you find the course useful?" into "which of these named competences did the course develop, and how?"

2. Ask students to evidence the skill, not just rate it. A self-rated "5/5 on critical thinking" is nearly worthless — it is subject to the Dunning-Kruger self-assessment problem. A description of a specific thing the student can now do that they could not before is evidence. The distinction between rating and evidencing is the entire difference between a survey that flatters a programme and one that can inform its redesign.

3. Close the loop with the demand side. Student self-report is necessary but not sufficient; it has to be triangulated with what employers and graduates say once skills meet the market. That is the argument for closing the employer feedback loop and for graduate tracer studies — the same triangulation logic that underpins all defensible evaluation.

Where Koji fits

Traditional SET tools were built to measure satisfaction with a teaching experience. Asking them to surface skill development is asking a thermometer to weigh something. Koji is built for the harder task.

• Evidence, not just a rating. Koji's AI-moderated conversational interview can probe a self-assessed skill claim — "you said the project improved your data-analysis ability; walk me through what you can now do" — turning an inflated Likert response into a concrete, analysable account. This is the single most important capability for skills-aware evaluation, and it is precisely what a static form cannot do.
• Structured mapping to competence frameworks. With six question types (open_ended, scale, single_choice, multiple_choice, ranking and yes_no), a programme can align items to ESCO or DigComp/EntreComp/GreenComp constructs and ask students to rank which competences a course actually developed, not merely rate satisfaction.
• Thematic analysis across the cohort. Koji's automatic thematic analysis aggregates those individual accounts into programme-level evidence of which skills are developing and which are not — the kind of accreditation-relevant, demand-aligned evidence a satisfaction mean cannot provide — with programme- and institution-level reporting to match.

Koji does not claim to measure employability; nothing honestly can, and we say so plainly. It claims to measure the proximal skill development that sits on the causal path to employability, in a form rich enough to be evidence rather than optimism. The same conversational engine on the main Koji platform does exactly this for companies researching customer needs — probing a stated preference for the behaviour behind it.

The bottom line

The labour market is shifting toward skills, slowly and amid a great deal of hype. The right response for programme evaluation is neither to ignore it nor to slap an "employability" number on the end of a satisfaction survey. It is to measure what a programme can actually influence and be held to — the development of named, transferable skills — with instruments rich enough to distinguish a real capability from a confident self-rating, and to triangulate that with the employers and graduates who see the skills in action. Evaluating the credential experience was the old question. Evaluating whether the programme built what the credential is supposed to stand for is the new one.