A 4.5 Won't Tell You Your Graduates Can't Get Hired: Evaluating Computer Science Programmes

A computer science course can score 4.5 out of 5 and still be failing its graduates. Students rate the lecturer who is clear, entertaining, and generous with marks — and those ratings tell you almost nothing about whether the curriculum still matches what employers need, whether the assessment produced demonstrable competence, or whether the people who left with a first can actually get hired. In no discipline is the gap between satisfaction and outcome wider, or more consequential, than in computing.

If you evaluate a CS programme with an end-of-module satisfaction survey, you are measuring the one thing that matters least.

The paradox that breaks the satisfaction survey

Computing sits on top of a genuine contradiction. Europe has a severe, well-documented shortage of technology talent: the EU's Digital Decade target is 20 million ICT specialists in employment by 2030, but the State of the Digital Decade assessment reports the bloc only crossed roughly halfway — about 10.5 million ICT specialists in 2025 — and, at the current rate, will fall short of the target by some five million workers. More than half of EU companies report difficulty finding staff with the right digital skills.

And yet — at the very same time — recent computer science graduates are finding the entry-level market unusually hostile. Analysis from the Federal Reserve Bank of New York's tracking of the labour market for recent college graduates has shown recent-graduate unemployment for computer science running elevated relative to many other fields, a signal that even a discipline long treated as a guaranteed path to work is under pressure. (That is US data, but the structural story — automation of junior tasks, employers wanting demonstrated skill over a certificate — is not confined to one continent.)

A shortage of specialists alongside unemployed graduates is not a contradiction the labour market will resolve on its own. It is a quality signal aimed straight at higher education: the problem is not the number of computing graduates but the match between what they can demonstrably do and what employers will pay for. Europe's own pipeline data underlines the stakes — in 2023 just 2.7 of every 1,000 young people in the EU graduated in ICT, against 3.7 in the United States and 4.6 in the United Kingdom.

None of that appears in a satisfaction mean. A survey item — "the module was well organised: 4.4" — cannot see obsolescence, cannot see the employer gap, and cannot see whether a graduate is job-ready. It measures the experience of consumption, not the value of the outcome.

Two things a satisfaction score structurally cannot detect

Curriculum obsolescence. Computing content decays faster than almost any other discipline's. A framework, a language idiom, or a deployment practice taught in first year can be legacy by graduation. Students are poorly placed to notice — they cannot rate a curriculum against an industry they have not yet entered, so a course teaching outdated tooling can still score highly on clarity and organisation. The people who can see the gap are employers and recent alumni, and neither is in your end-of-term survey population. This is why retrospective, alumni-based evaluation carries information the in-course survey never will.

Demonstrated competence. European accreditation for computing has already moved to an outcomes basis. The Euro-Inf Quality Label, awarded by EQANIE, certifies informatics degrees against a Europe-wide framework of programme learning outcomes — recently updated to keep its structure while sharpening outcomes and adding sustainability, accessibility, privacy, and compliance — and professional bodies such as the BCS accredit on a similar competence logic. Like EUR-ACE for engineering, these frameworks ask for evidence of what graduates can do. A Likert average is not that evidence; it is an indirect measure of experience standing in for a direct measure of capability the accreditor actually wants.

But isn't student satisfaction still worth measuring?

Yes — and this is the honest qualification. Satisfaction is not worthless; it is just misassigned. A cohort that is confused, unsupported, or demoralised will learn less, and student experience is a legitimate object of concern in its own right. The mistake is not measuring satisfaction; it is treating satisfaction as a proxy for programme quality in a field where the two routinely diverge.

There is a second fair objection: employers are a moving target. Chase every framework fad and you produce graduates trained for last year's job posting, not thinkers who can adapt. That critique is correct, and it is an argument against evaluating a CS programme by narrow tool-coverage checklists — but it is not an argument for the satisfaction survey. It is an argument for evaluating the durable outcomes accreditation actually names: problem-solving, the ability to learn new tools, and professional judgement. Those, too, are invisible to a mean. The resolution is not to stop asking students; it is to ask them, and their employers and alumni, questions a scale cannot answer.

Where Koji fits: outcome-aware evaluation for computing

The reason CS evaluation defaults to a satisfaction mean is that the alternative — capturing what students learned to do, where the curriculum felt dated, which skills they could not evidence at interview — has historically been too labour-intensive to collect at programme scale. That is the constraint Koji for Education removes.

Koji runs AI-moderated conversational interviews that probe past the rating: when a student flags an assessment or a topic, the moderator follows up for the specific, actionable detail a checkbox discards. Automatic thematic analysis turns hundreds of open-text responses across a programme into named, ranked issues — "the databases module taught deprecated tooling," "no student could evidence version-control competence" — instead of an unread comment dump. Its six structured question types let you map feedback directly onto Euro-Inf-style outcome statements rather than generic satisfaction items, and quality scoring keeps low-effort responses from polluting the signal. Because collection can run formatively and mid-cycle, obsolescence can be caught and corrected within a delivery, not a validation cycle. And the standardised, bias-aware AI moderator makes the evidence consistent across a large, modular CS programme — the kind of traceable, outcome-referenced evidence an accreditor recognises. Koji surfaces and helps you act on the outcome gap; it does not claim to eliminate bias or guarantee employability, and no tool honestly can.

A 4.5 is a comfortable number. In computing, it is also the number most likely to hide the graduates who cannot get hired. (Departments running general user or product research can use the same interview engine on the main Koji platform.)

Evaluating a computing programme against Euro-Inf or BCS outcomes? See how Koji for Education captures demonstrated-competence evidence, not just satisfaction.

Frequently asked questions

Why is student satisfaction a poor measure of computer science programme quality?

Because computing's value lies in outcomes — demonstrable competence and employability — that a satisfaction score cannot see. A module can score highly for clarity while teaching obsolete tooling, and students cannot rate a curriculum against an industry they have not yet entered.

What is the Euro-Inf Quality Label?

Euro-Inf is a European quality label for informatics and computing degrees, awarded by EQANIE against a Europe-wide framework of programme learning outcomes. Like EUR-ACE in engineering, it asks for evidence of what graduates can do, not indirect satisfaction measures.

Is there really a shortage of tech workers if CS graduates struggle to find jobs?

Both are true simultaneously. The EU targets 20 million ICT specialists by 2030 but is on track to fall roughly five million short, while recent CS graduates face an unusually tough entry-level market. The signal is about the match between demonstrable skills and employer needs, not raw graduate numbers.

How should a computing programme be evaluated instead?

By combining student feedback with outcome-referenced evidence: what students can demonstrably do, where the curriculum felt dated, and what employers and alumni observe — mapped onto accreditation outcome statements rather than generic satisfaction items.

How does Koji support outcome-aware CS evaluation?

Koji's conversational interviews probe for actionable specifics, its thematic analysis turns programme-scale open text into ranked issues, and its structured question types let feedback map onto Euro-Inf-style outcomes — producing demonstrated-competence evidence rather than a satisfaction mean.