Employers Have Repriced AI Skills. Your Course Evaluation Has Not Noticed.

The labor market has repriced AI competence faster than any skill in recent memory. In the 2024 Microsoft–LinkedIn Work Trend Index — a survey of 31,000 people across 31 countries — 66% of leaders said they would not hire someone without AI skills, and three in four knowledge workers already used generative AI at work. The World Economic Forum's Future of Jobs Report 2025 finds employers expect 39% of workers' core skills to change by 2030, with AI and big data the single fastest-growing skill category. Meanwhile, the median European course evaluation questionnaire — often unchanged in a decade — asks students about clarity, workload and satisfaction, and precisely nothing about whether the programme built the competences this labor market now screens for. If course and programme evaluation is supposed to generate evidence about educational effectiveness, it is now systematically blind on the dimension employers reprice fastest.

The scale of the repricing

Treat vendor and consultancy numbers skeptically; these two sources survive scrutiny better than most because of their sample sizes and methodological transparency, and they agree on direction:

• Hiring signals. In the Work Trend Index 2024, 66% of leaders would not hire a candidate without AI skills, and 71% said they would rather hire a less experienced candidate with AI skills than a more experienced one without — an extraordinary inversion of the experience premium. At the same time only 39% of AI users had received any employer training, meaning employers are importing, not building, this competence — largely from education systems and self-teaching.
• Skill-demand trajectories. The WEF's 2025 employer survey (over 1,000 employers representing 14 million workers) puts AI and big data at the top of the fastest-growing skills to 2030, alongside networks, cybersecurity and technological literacy, within a broader forecast of 170 million roles created and 92 million displaced.

You can quarrel with any single estimate. You cannot reasonably read this evidence and conclude that a graduate's AI-related competence is irrelevant to the employability outcomes universities are increasingly held accountable for.

The measurement gap

Now look at the instrument. Standard end-of-course questionnaires measure perceived teaching quality and satisfaction. Even institutions that have moved beyond satisfaction toward skills measurement rarely include a single item on whether students learned to use, evaluate, or critically judge AI tools within their discipline — despite most European institutions now having position statements on AI in teaching.

This creates three specific evidence failures:

1. Curriculum committees fly blind. Programmes are redesigning around AI at speed — but without evaluation items on AI-related learning, review panels have no student-experience evidence on whether integration is working, deferred instead to anecdote and enthusiasm.
2. The confound goes unmeasured. Students now use generative AI throughout their coursework, which distorts workload and difficulty ratings in ways invisible to instruments that never ask about AI use.
3. Employability claims lose their chain of evidence. Institutions tell accreditors and applicants that programmes build future-ready skills. Skills claims need proximal mediators measured during the programme, not just tracer studies years later — and AI competence currently has no proximal measurement at all in most evaluation systems.

What to measure — without chasing hype

The objection writes itself, so let us state it at full strength: universities should not whipsaw curricula toward every technology employers panic about; education builds durable capacities, not tool fluency that depreciates in eighteen months. Three responses.

First, the durable-capacity argument is an argument about what to measure, not whether. The competences that survive tool churn — knowing when AI output is trustworthy, what to delegate versus verify, how to work with machine-generated drafts critically, data judgment — are classical critical-thinking capacities expressed in a new medium. Europe already has a vocabulary for this: DigComp, whose mapping to course evaluation we have covered, treats critical evaluation of information and tools as core digital competence, not vendor training.

Second, the skill-change number cuts both ways: if 39% of core skills change by 2030, curricula anchored to any static skill list — including yesterday's — are the risk. Measurement is how you notice drift without panic.

Third, self-reported competence is indeed weak evidence — students notoriously misjudge their own learning. The answer is item design and triangulation, not silence: ask about concrete practice ("in this course, did you evaluate AI-generated output against primary sources?") rather than confidence ("rate your AI skills"), and read results alongside assessment evidence, not instead of it.

Practically, programme teams should add a small, stable block to course and programme evaluation:

• Exposure: where in the course students used or were taught to use AI tools, if at all — including whether use was prohibited, tolerated or designed-in.
• Critical practice: behaviorally anchored items on verifying, critiquing and correcting AI output within the discipline.
• Transfer confidence, carefully framed: whether students can articulate where AI helps and fails in their field — the kind of open-ended question that separates real learning from vibes.
• Programme-level synthesis: aggregated across courses, mapped against the programme's own stated AI-integration goals, feeding programme review and the employer feedback loop so employer signals and student evidence confront each other rather than passing in the night.

Sequencing it: a realistic first cycle

For a programme team starting from zero, the credible path is incremental. Cycle one: add three items to existing course evaluations — one exposure item (single choice: how AI figured in this course), one critical-practice item (yes/no with follow-up: did you verify or correct AI output against course sources), one open-ended transfer question (where does AI help and mislead in this field). Deliberately small: three items will not aggravate survey fatigue, and they generate a baseline. Cycle two: review theme distributions with programme committees, refine wording where students misread items, and set programme-level targets tied to stated AI-integration goals. Cycle three: bring employer-side evidence into the same review — advisory boards, placement supervisors, graduate destination signals — and reconcile where student experience and employer demand diverge. Three cycles is eighteen months; an institution that starts this autumn has a defensible evidence trail before most competitors have agreed on a working definition of AI literacy. The alternative — waiting for a sector-standard instrument to emerge — cedes exactly the years in which the labor market is moving fastest.

Where Koji fits

This measurement problem is awkward for legacy tools precisely because it is new, evolving, and partly qualitative — the conditions under which static annual questionnaires perform worst.

• Structured flexibility. Koji's six question types (open-ended, scale, single and multiple choice, ranking, yes/no) let programme teams add a compact AI-competence block without rebuilding the instrument, and iterate it as the technology moves — including mid-cycle, since Koji supports formative in-semester collection rather than one retrospective snapshot.
• Conversational depth where it matters. The difference between a student who genuinely learned critical AI practice and one who pasted prompts is invisible to a Likert item. Koji's AI-moderated interviews probe: which tasks, what went wrong, how they verified. That is exactly the evidence curriculum committees currently lack.
• Thematic analysis across the programme. Open-text responses about AI use are automatically organized into themes with quality scoring, so a programme director sees where AI integration is producing learning and where it is producing shortcuts — across every course, not the two whose instructors volunteered anecdotes.
• Closing the loop. Findings feed action tracking and programme-level reporting, giving accreditation panels a documented chain from employer signal → curriculum change → student evidence.

Teams on the employer side of this loop — running skills research with graduates, hiring managers or customers — use the same AI interview engine on the main Koji platform.

The takeaway

The employer repricing of AI competence is among the best-documented labor-market shifts of the decade: two-thirds of leaders screening hires for AI skills, AI and big data the fastest-growing skill demand to 2030, and 39% of core skills in motion. Programme evaluation does not get to sit this out. The task is not to bolt marketing language onto questionnaires; it is to measure exposure and critical practice with well-designed items, triangulate against assessment, and connect the results to employer feedback at programme level. Institutions that start now will have three years of trend evidence when their accreditor — or their applicants — start asking.

Koji for Education makes evaluation instruments that can keep up: conversational, structured, thematically analyzed, and built for programme-level evidence. Book a demo.