The Market Already Grades Your Law Degree — and It Grades Competence, Not Contentment

A law degree has an unusually honest external examiner: the professional qualification that follows it. In England and Wales the Solicitors Regulation Authority's Solicitors Qualifying Examination (SQE), the Bar's assessments and pupillage, and on the continent the state examinations decide whether your graduates can actually practise. None of them ask whether students enjoyed the module. If your programme evaluation stops at a satisfaction mean, you are measuring the one variable the people who license your graduates have deliberately excluded.

That is not an argument against course evaluation. It is an argument against a particular kind of course evaluation — the end-of-module Likert average — being treated as the verdict on a professional programme whose real verdict is delivered elsewhere, later, and against a competence standard your survey never mentions.

The qualification is the real evaluation

Since 2021 the route to becoming a solicitor in England and Wales runs through the SQE: a two-part, centralised, criterion-referenced assessment of legal knowledge (SQE1) and practical legal skills (SQE2), plus two years of Qualifying Work Experience and a character-and-suitability check (Solicitors Regulation Authority). Crucially, a law degree is no longer a formal gateway: candidates may hold a degree in any subject. The competence gate has been decoupled from the classroom and handed to a standardised national examination.

And that gate is unforgiving. First-time SQE1 pass rates have ranged from roughly 46% to 60%, and the July 2025 sitting fell to a record low of 41% (Legal Cheek). SQE2, the skills assessment, hit a record high of 82% in the April–May 2025 sitting, with first-time sitters at 84% (Legal Cheek). Whatever else these numbers are, they are an external, criterion-referenced grade of competence — the thing accreditors call a direct measure of learning. Your module satisfaction mean is an indirect measure at best, and on the SQE's own terms, irrelevant.

Europe grades competence too — often harder

The English decoupling is a recent, visible example of a pattern that runs across European legal education: the licence to practise is a competence gate in which student satisfaction plays no part.

Germany's Einheitsjurist model is the sharpest case. A law graduate must pass the First State Examination, complete a two-year practical traineeship (the Referendariat) rotating through civil and criminal courts, an administrative body and a law firm, and then pass the Second State Examination — a comprehensive assessment of whether the candidate can actually function as a judge, prosecutor or advocate (Freie Universität Berlin). France routes graduates through the CRFPA entrance examination and the CAPA; other member states impose their own bar examinations and traineeships. The common structure: theory, supervised practice, and a terminal competence assessment. Nowhere does a happiness score enter the judgement.

This is the same argument we have made about regulated-profession programmes under Directive 2005/36/EC and about actuarial degrees the market grades before it asks about satisfaction: when an external body owns the definition of competence, an internal satisfaction metric is not the outcome — it is, at most, an early-warning system for the outcome.

What a satisfaction mean cannot see

Legal education produces things a five-point scale is structurally unable to detect. Can the student construct a valid argument from authority? Draft a contract clause that survives contact with a counterparty? Spot the ethical conflict that is not flagged in the question? Hold a line of advocacy under pressure? Behave as a member of a regulated profession with duties to the court and the client?

These are the constructs the SQE, the Bar and the state examinations assess. A module survey asking whether the lecturer was clear and the reading list manageable measures something real — but it is a different thing. In measurement terms this is construct underrepresentation: the instrument samples a narrow slice (delivery, workload, clarity) of a much wider competence domain, then reports the average as if it spoke for the whole. Law programmes also do something a Likert item cannot register at all: they form a professional identity — the shift from "student who knows law" to "person who acts as a lawyer".

But doesn't the SQE pass rate already tell us everything?

The strongest counterargument deserves a straight answer. If an external, standardised competence exam grades your graduates, why evaluate the course at all — why not just watch the pass rate?

Three reasons. First, timing: SQE and state-examination results arrive one to three years after the teaching that shaped them. By the time a cohort's pass rate lands, the module has run two or three more times. Pass rates cannot drive the mid-module correction that actually improves teaching — the evidence is clear that course evaluation only improves teaching when it feeds timely consultation, not a retrospective audit. Second, confounding: pass rates are contaminated by prior attainment, by who could afford an expensive commercial prep course, and by selection into the exam. A high pass rate can reflect a well-resourced intake rather than good teaching; a dip can reflect a widened-access cohort you should be proud of. Third, Goodhart's Law: make the SQE pass rate the target and programmes will teach to it, narrowing legal education to exam technique — which is precisely the pathology the SRA's own reviewers worry about.

So the pass rate is necessary but not sufficient. What a professional law programme needs alongside it is formative, competence-referenced evidence gathered while the teaching is still live — evidence about whether students can reason, draft and argue, not merely whether they were content.

What competence-aware evaluation looks like

This is where the instrument has to change, not just the questions. Koji for Education replaces the static end-of-module form with AI-moderated conversational interviews that probe what students can do: instead of "Rate the seminar 1–5", the interview asks a student to describe how they approached a problem question, where their reasoning broke down, and what scaffolding would have helped — then follows up on the answer. Its six structured question types (open-ended, scale, single- and multiple-choice, ranking, yes/no) let a programme ask students to rank which competences they feel least prepared for, while automatic thematic analysis turns hundreds of open-text answers about drafting or advocacy into themes a programme director can act on, each traceable to the quote behind it.

Because the moderation is standardised and bias-aware, a programme can run the same probing evaluation across every module without the inconsistency of different human moderators — and run it mid-cycle, not just at the end, so that a cohort struggling with legal method gets help before the SQE, not a post-mortem after it. Programme- and institution-level reporting lets you align the evidence to the SRA's Statement of Solicitor Competence or the national exam's competence framework rather than to a generic satisfaction template, and closing-the-loop tracking records what you changed in response. Many law schools also run wider empirical and access-to-justice research; the same AI interview engine underpins the general-purpose koji.so platform for that work. Throughout, data handling is GDPR/AVG-compliant and EU-appropriate — a non-trivial point for a discipline that teaches data protection.

None of this eliminates the limits of self-report — students still cannot fully judge their own competence, and the professional examination remains the arbiter. But it surfaces, early and in the students' own words, the competence gaps that a satisfaction mean smooths over.

The bottom line

Your law degree already has a brutally honest evaluator: the qualification that decides whether your graduates can practise. Stop asking your course evaluation to duplicate the satisfaction question the SQE and the state examinations pointedly refuse to ask. Ask it instead to give you the competence signal those examinations deliver too late — while you can still act on it.

See how Koji for Education helps law programmes evaluate competence, not just contentment — start with Koji for Education.

Frequently asked questions

Does the SQE mean a law degree no longer matters for becoming a solicitor?

Not exactly. The SRA no longer requires a Qualifying Law Degree — candidates can qualify with a degree in any subject plus the SQE and two years of Qualifying Work Experience. But most candidates still take a law degree or conversion course to prepare, and the degree remains the main site where legal reasoning and professional identity are formed. What changed is that the competence gate is now a standardised national examination, not the degree itself.

Why not just use SQE and Bar pass rates as our course-evaluation metric?

Pass rates are a valuable external, criterion-referenced outcome, but they arrive one to three years late, are confounded by prior attainment and access to expensive prep courses, and cannot drive mid-module improvement. Used as a target they also invite teaching-to-the-exam. They are necessary but not sufficient; formative, competence-referenced evaluation during the programme complements them.

What can a satisfaction survey not measure in a law programme?

It cannot directly measure legal reasoning, drafting quality, advocacy, ethical judgement, or the formation of a professional identity — the constructs the SQE, the Bar and continental state examinations actually assess. A Likert average captures delivery, clarity and workload, which is a narrow slice of the competence domain (construct underrepresentation).

How is legal education assessed elsewhere in Europe?

Most European systems gate the licence to practise with a competence assessment rather than satisfaction. Germany's Einheitsjurist model requires two State Examinations and a two-year Referendariat traineeship; France uses the CRFPA and CAPA; other states impose bar examinations and supervised practice. Satisfaction plays no formal role in any of them.

How does Koji evaluate competence rather than satisfaction?

Koji runs AI-moderated conversational interviews that probe what students can do — how they reasoned through a problem, where they struggled, what would have helped — and uses structured question types plus automatic thematic analysis to turn open-text answers into actionable, quote-traceable themes. It supports formative mid-cycle collection and programme-level reporting aligned to competence frameworks, with GDPR/AVG-compliant data handling.

Can course evaluation predict who will pass the SQE?

It should not try to. Predicting individual pass probability from self-report is unreliable and ethically fraught. The useful role of formative evaluation is diagnostic, not predictive: surfacing which competences a cohort feels least prepared for, early enough to change teaching before the professional examination.