Evaluating Professional Accountancy Programmes: The Exam Passes, But Does the Evaluation See the Competence?

Bottom line up front: A professional accountancy programme is not certified by its exam pass rate, and it is certainly not certified by student satisfaction. Under the international framework that governs the profession, a qualified accountant is produced by the combination of examined knowledge and a multi-year period of supervised, assessed practical experience. Any course evaluation that stops at "were students happy with the module" is measuring a thin slice of a much longer competence pipeline — and often the least predictive slice. Evaluating accountancy education well means aligning your evidence with the competence framework the profession is actually held to.

What the profession is really certifying

The global baseline for accounting education is the International Federation of Accountants (IFAC) suite of International Education Standards (IES). These separate examined technical competence (IES 2), professional skills (IES 3), values and ethics (IES 4) and — critically — practical experience (IES 5). Professional bodies build their qualifications on this scaffolding. The ACCA Practical Experience Requirement, for example, requires 36 months of relevant work and the achievement of nine performance objectives, each signed off by a qualified supervisor — a design derived directly from IES 5.

For statutory auditors the requirement is not merely a professional-body rule but EU law. The Statutory Audit Directive (2006/43/EC), Article 10 requires a trainee to complete a minimum of three years of practical training, of which at least two-thirds must be with an approved statutory auditor or audit firm, before approval. So the thing society values — a person who can be trusted to sign an audit opinion — is defined by a competence-and-experience standard that a university module survey does not, and structurally cannot, observe.

Why the usual evidence misleads

Accountancy programmes tend to lean on two evidence sources, and both mislead in predictable ways.

Exam pass rates. Pass rates are a genuine signal, but a dangerous one to optimise. As soon as a pass rate becomes the target by which a programme is judged, it stops being a good measure — the familiar logic of Goodhart’s Law in evaluation. Teaching narrows toward what the exam rewards: recall and question-drilling, at the expense of the professional judgement, scepticism and ethics (IES 3 and IES 4) that the exam samples only lightly and that practice most demands. A programme can lift pass rates and still graduate technicians who freeze the first time a client applies pressure to a judgement.

Satisfaction scores. End-of-module satisfaction is even weaker here. Professional accountancy students are frequently working, fee-paying and instrumentally focused; their satisfaction tracks logistics, exam alignment and perceived pass-likelihood far more than it tracks whether the programme is building durable professional competence. Worse, the demanding, ambiguity-tolerant teaching that best prepares a trainee auditor — cases with no clean answer, ethics dilemmas, feedback that stings — often depresses short-run satisfaction. Reward the score and you quietly punish the pedagogy.

The evidence that is actually missing

The gap in almost every accountancy programme’s evaluation is the employer and workplace-supervisor voice, and the retrospective view from students once they are in practice. IES 5 is fulfilled in the workplace, assessed by a workplace supervisor — yet that supervisor is rarely, if ever, asked what the academic programme did or did not prepare the trainee to do. The people best placed to judge whether a curriculum builds real competence are structurally excluded from the evaluation of it. Closing that loop is the single highest-value move an accountancy programme can make, and it is the theme we develop for professional education generally in closing the employer feedback loop.

Equally, the honest verdict on a professional programme often only becomes visible in delayed hindsight. A module that felt like pointless theory in week six is frequently the one a trainee blesses two years later when it turns out to underpin a real decision — the alumni retrospective signal that in-course surveys cannot capture.

"But the professional body already assures quality"

The strongest counterargument is that accreditation by ACCA, ICAEW, CIMA, Chartered Accountants Ireland or an equivalent body already assures quality, so institutional evaluation is redundant. This confuses two different jobs. Accreditation confirms that a programme covers the syllabus and meets minimum standards at the point of review, on a multi-year cycle. It does not tell a programme director, this semester, which module is failing to build professional scepticism, which cohort feels unprepared for their first busy season, or which employer partner is quietly disappointed. Accreditation is a periodic floor; continuous, competence-aligned evaluation is how you stay well above it — and how you generate the very evidence the next accreditation visit will demand. The two are complements, not substitutes, in the same way we describe for assurance of learning under AACSB, EQUIS and AMBA.

The part the exam barely tests

The competences most predictive of a safe, trusted accountant are the ones a written exam samples least well: professional scepticism, ethical courage under commercial pressure, and sound judgement in ambiguity — the domains IES 3 and IES 4 place at the centre of the profession. These are hard to examine and easy to leave unevaluated, so programmes drift toward measuring what is convenient rather than what is decisive. A student can score highly on a technical paper and still fold the first time a partner or client leans on them to see a number differently. If your evaluation never asks whether students felt equipped to hold a line — to say no to a manager, to escalate a concern, to sit with an unresolved judgement — then you are not evaluating the qualities the public actually relies on the profession to guarantee. And because those qualities surface in conduct rather than on a script, the only way to evaluate them is to ask students and their supervisors about real situations they faced — not to infer them from a satisfaction mean or a pass rate.

The apprenticeship route changes the evidence problem

Increasingly, professional accountancy is qualified through work-integrated routes — degree and higher apprenticeships that interleave study with employment from day one — rather than the classic study-then-train sequence. This dissolves the tidy boundary between academic programme and workplace, and it makes single-source student surveys even less adequate: the employer is now a co-educator, and any honest evaluation has to be tripartite, hearing from the apprentice, the training provider and the employer together. We develop this three-way evidence model in evaluating degree apprenticeships; accountancy is one of its most important applications, because the qualification’s credibility depends on the workplace half of it being evaluated as seriously as the classroom half.

How Koji fits

Koji for Education lets an accountancy programme evaluate against the competence framework it is actually held to, not just the exam it teaches toward. Its six structured question types and AI-moderated conversational interviews can probe professional judgement, ethics confidence and preparedness — "describe a time on placement you had to push back on a manager; did your studies prepare you for that?" — rather than reducing competence to a satisfaction number. The same standardised interview engine can be run with employers and workplace supervisors, turning the missing IES 5 voice into structured, comparable evidence, and with alumni for the delayed, higher-validity verdict. Automatic thematic analysis aggregates open-text feedback from students, employers and supervisors into programme-level themes, and closing-the-loop tracking records what the programme changed in response — exactly the documented, continuous-improvement evidence an accreditation panel wants to see. All of it is GDPR-compliant and EU-appropriate by design.

None of this replaces examinations, workplace supervisors or professional-body judgement; it mitigates the blind spots of exam-and-satisfaction evaluation and surfaces competence signals those instruments miss. Accountancy schools that also run wider stakeholder or graduate research can use the shared AI interview engine behind koji.so.

If you lead an accountancy or professional finance programme and your evaluation still stops at pass rates and satisfaction, see how Koji for Education helps you evaluate the competence the profession actually certifies.