Ofqual at Education Committee

The independent regulator favoured maintaining exams, either by using socially distanced approaches or postponing assessments till later in the year – but it was the calculated grade option that the Secretary of State preferred.

To be fair, at the time these options were offered it was not clear for how long schools would be closed. Clearly, viewing the spring and summer of 2020 as it played out, it is clear that either of the alternate options would have caused more problems.

Ofqual has an advisory role, a duty to carry out the policies of ministers, and statutory duties set out in the Apprenticeships, Skills, Children and Learning Act 2009

• To secure qualifications standards.
• To promote National Assessment standards.
• To promote public confidence in regulated qualifications and National Assessment arrangements.
• To promote awareness of the range and benefits of regulated qualifications.
• To secure that regulated qualifications are provided efficiently.

Importantly, these statutory requirements allow directions from ministers not to be carried out if they are in conflict, which explains the same story of the appeal guidance issued and withdrawn on the same day. It turns out that Williamson’s (frankly) harebrained plan to use mock exams as a basis for appeals was not one that the regulator took delight in – and that the Saturday morning release was very much in the spirit of a least harmful approach to deliver the Minister’s policy (an idea familiar to anyone who has done time in an NDPB!).

The Secretary of State’s increasingly frantic calls to Ofqual Chair Roger Taylor and Chief Regulator Sally Collier over Saturday evening brought about an emergency weekend board meeting, and what was intended to be the temporary rescindment of the guidance while further discussions took place. These discussions seemingly convinced Williamson of the need for the infamous u-turn – with students being issued Centre Assessed Grades (CAGs) the following week.

And Roger Taylor had to threaten to resign if the Secretary of State did not have sufficient confidence in Ofqual. In Taylor’s words – “they discussed it, and resolved the issue”.

Look over there!

This, rather than the attempts by the committee to understand the algorithm (Michelle Meadow’s beautifully expressed “I don’t believe the algorithm ever mutated” offering a representative taste of the level of debate) was the story of the day. Each decision was taken by, or on behalf of, the Secretary of State – from the decision to use calculated grades to the ill-fated decision to allow appeals based on mock exams. Around 12 different algorithms were tested by Ofqual (using 2019 data) with the best chosen given the paucity of data available.

The repeated attempts of committee members to grasp the fact that Ofqual, in modelling the grades that a “normal” year would offer, would have to take into account the differential in educational and other opportunities enjoyed by the better off. In a nutshell, and as we’ve said on Wonkhe several times, A levels are biased – and the modelled grades reflected that.

On that basis – counter-intuitively – the initial algorithm was fair, in that it was a plausible probabilistic model of one of a range of potential outcomes to a putative 2020 exam series. As was pointed out seventy-five percent of students do not get their predicted grades in a normal year. Ofqual Chair Roger Taylor argued that the idea of grades within the calculations was perhaps unhelpful in itself:

If you can’t replicate normal grades, don’t pretend you can. The objective was to enable progress, not award GCE grades”

It’s clear now – in hindsight that any algorithm, not just the algorithm chosen, would have been fundamentally unacceptable to teachers, parents, and students. A point made clearly in the written statement provided to the committee.

The clarity of responses from Taylor, Meadows, new Chief Regulator Glenys Stacey, and Executive Director for General Qualifications Julie Swan belied some notably sharp questioning from Committee Chair Robert Halfon on Ofqual’s approach to communication. He conflated, perhaps to his disadvantage, the difficulty in getting a press response from Ofqual during the point of crisis with the “eleven, soon to be twelve” staff with communications responsibilities (though Stacey did note that many of these roles involved the production of required publications rather than dealing with the press).

It is difficult to agree that Ofqual is defacto “unfit for purpose” because of the events of 2020 – but we would perhaps all agree that there are lessons to be learned about the wisdom of ministers overruling the decisions of independent regulators.

Next year’s model

For 2021 Ofqual has recommended to the Secretary of State – once again – that exams should take place if at all possible. Committee members were exercised about the options where exams could not take place – Ofqual had a strong view that we should not return to centre assessed grades, given time to develop an alternative approach to exams.

However, there was very little on the disadvantages conferred on the 2021 cohort. A perfect storm of deferrals, uncertain lockdown status, and competing for university places with “better” (on paper) applicants from the class of 2020. There’s a lot more work to do, and there’s been no sign that anyone is starting to consider it.

But the wider implications – beyond the politics of it all – are fascinating. What are “accurate grades”? What would they look like? If the calculation was a proxy for examination, what is examination a proxy for? How accurate, for that matter, are examinations in doing whatever it is they do?

And how much use is any of it for understanding which students are likely to succeed in which courses of higher education?

Things to ponder, perhaps, as we await the promised release (by Ofqual, in consultation with DfE) of minutes and correspondence detailing the events of summer 2020.