Evaluating Programmes Against Jobs That Do Not Exist Yet: Cedefop, Skills Anticipation, and Course Evaluation

Bottom line up front: Most course evaluation is retrospective by construction — it asks students how satisfied they were with a module that has already ended. But programme quality is increasingly judged against a forward question: are graduates ready for a labour market that is still forming? Europe has a serious tool for that forward view — Cedefop's Skills Forecast, which projects employment and skill demand to 2035 as an explicit "early warning mechanism". The opportunity is to make course evaluation partly anticipatory: map what a course teaches to where the field is heading, and ask students whether they feel prepared for that, not only whether they enjoyed it. The catch is equally important — skills forecasts are probabilistic and macro-level, and student self-assessment of future-readiness is weak evidence. So the honest move is triangulation, not substitution.

The retrospective trap

A standard end-of-module survey is a satisfaction snapshot of something finished. That has value, but it structurally cannot tell you whether the programme is preparing students for the roles they will actually enter. Graduate-outcome metrics — earnings, employment, destination — try to close that gap, but they arrive years too late to steer a curriculum, which is the leading-versus-lagging-indicator problem at the heart of employability evidence. Course evaluation is the only feedback you collect while you can still change something. The question is whether it can be pointed forward as well as backward.

What Cedefop actually offers

Cedefop — the European Centre for the Development of Vocational Training — produces quantitative projections of future employment by sector and occupation across the EU-27 plus Norway, Iceland, Switzerland, North Macedonia and Türkiye, with the latest Skills Forecast extending to 2035. It is designed as an early-warning system for labour-market imbalances, and its recent findings are pointed: even as the supply of highly educated workers rises, demand is shifting unevenly, so Cedefop projects the underutilisation of high-skilled graduates coexisting with persistent vacancies elsewhere and widening qualification mismatches. Its Skills in transition analysis maps how tasks within occupations — not just occupation counts — are changing, which is the more useful grain for curriculum design.

This is the raw material for a forward-looking evaluation question. It gives you a defensible, external, comparable evidence base for what "preparedness" should mean in a given field, rather than leaving it to a programme team's intuition or an employer's most recent complaint.

How to make evaluation anticipatory without breaking it

The practical method has three moves. First, map course learning outcomes to anticipated skill demand — the same mapping discipline we describe for the ESCO skills taxonomy, but using Cedefop's directional signals about which skill clusters are growing, shrinking, or being reshaped. Second, add a small number of forward-looking items to evaluation: not "was the content up to date?" but "how prepared do you feel for where this field is heading?" and, crucially, open-ended probes into why. Third, combine student signal with real-time labour-market data — the job-postings and labour-market intelligence that show what employers are asking for now, which Cedefop's medium-term forecast contextualises. Together these let a programme triangulate three time horizons: the immediate market (postings), the medium-term forecast (Cedefop), and the student's own sense of preparedness (evaluation). None is sufficient alone. This is the same anticipatory logic behind the EU Union of Skills agenda that Cedefop's data underpins, and it sharpens the perennial skills-gap conversation that AI is now accelerating in fields like technology and analytics.

But doesn't this just chase forecasts and distrust students?

Two strong objections deserve a straight answer.

"Forecasts are guesses — you'll rebuild your curriculum around a projection that turns out wrong." Partly fair. Cedefop is explicit that its output is a scenario-based projection, not a prophecy, and the history of skills forecasting is littered with confident predictions the market ignored. The defence is to use forecasts as directional priors, never as targets. A projection that a skill cluster is growing is a reason to ask a sharper evaluation question about it — not a mandate to reweight the whole programme. Treating a forecast as a target invites the Goodhart's-law failure where you optimise the proxy and lose the thing. The forecast informs the question; the evidence still decides the answer.

"Students can't assess their own future-readiness." Also fair, and important. Self-assessed preparedness is a perception, and perceptions of one's own competence are systematically unreliable — the Dunning-Kruger problem in self-assessment is real. A confident graduate is not a prepared one. This is exactly why the forward-looking item must never stand alone: it is one leg of a triangle whose other legs are objective (labour-market data, employer input, direct assessment of demonstrated skills). The value of the student signal is not that it measures readiness accurately; it is that it surfaces where students think the gap is, which is a leading indicator of engagement and a prompt for investigation — and it is systematically distorted by the gap between what students and employers perceive as important, which is itself worth measuring.

Where Koji fits

The anticipatory question is only as good as the follow-up. "How prepared do you feel for where this field is going?" on a five-point scale produces a number that tells you almost nothing. Koji for Education runs AI-moderated conversational interviews that probe the why behind a rating — when a student says they feel underprepared, the AI asks in what respect, against which kinds of roles, and what would have helped. Its automatic thematic analysis then aggregates hundreds of those open-ended answers into structured, quotable themes that a programme team can map against Cedefop's skill clusters and against live labour-market data. That turns a vague anxiety about "future skills" into a specific, evidence-backed list of where students perceive the curriculum lagging the market — the input a curriculum review actually needs.

Koji does not forecast the labour market, and it does not pretend a student's self-assessment is objective truth. What it does is make the student leg of the triangulation rigorous — deep, thematic, and comparable across cohorts — so it can be set honestly against the harder data. Institutions running broader workforce or user research can use the same conversational interview engine on the main Koji platform.

Making course evaluation forward-looking is not about predicting the future. It is about asking students a better question, contextualised by the best available projection of where their field is going, and then refusing to trust any single source. Cedefop supplies the map. The evaluation supplies one of the three readings you take from it.

Frequently asked questions

What is the Cedefop Skills Forecast? It is the European Union''s medium-term projection of employment and skill demand by sector and occupation, produced by Cedefop for the EU-27 and several associated countries, currently extending to 2035. It is designed as an early-warning mechanism for labour-market imbalances and skills mismatches.

Can course evaluation really be forward-looking? Partly. You can add items about perceived preparedness for where a field is heading, and map course outcomes to anticipated skill demand. But student perceptions are not objective measures of future-readiness, so forward-looking evaluation only works as one leg of a triangulation with labour-market data and direct assessment.

Should we redesign our curriculum around a skills forecast? No. Cedefop is explicit that its output is a scenario-based projection, not a prediction. Use forecasts as directional priors that inform which questions to ask, not as targets to optimise — optimising a forecast is a classic Goodhart''s-law trap.

Why not just use graduate-outcome data instead? Graduate outcomes (earnings, employment, destinations) are lagging indicators that arrive years after the course, too late to steer the curriculum for current students. Forward-looking evaluation, combined with skills anticipation, is one of the few leading indicators available while change is still possible.

How is this different from mapping to ESCO or DigComp? Taxonomies like ESCO and DigComp describe skills that exist now; the Cedefop forecast adds a directional, time-dependent signal about which skills are growing, shrinking, or being reshaped. The two are complementary — map to the taxonomy, weight by the forecast.