The sector has been trying to make sense of the Teaching Excellence and Student Outcomes Framework (TEF) ever since it was first announced. But the Pearce Review of TEF – a statutory requirement inserted in the Higher Education and Research Act to placate the House of Lords – has led to a wait-and-see attitude in the sector.
TEF had already been paused for a year, to allow the final touches to be put on a subject-level TEF which would start in 2021. But this restart had been in some doubt, as various rumours regarding Dame Shirley Pearce’s review and the weight accorded to the elegantly devastating critiques from the Royal Statistical Society had led us to believe that a more fundamental revision was on the cards.
As my colleague Jim Dickinson notes, the confirmation of the 2021 start date for Subject TEF was unexpectedly restated in Gavin Williamson’s recent guidance letter to the Office for Students. A minister announcing something that has already been announced wouldn’t often raise eyebrows, but this cuts across the Pearce Review – which we can expect to see, alongside a government response later this year. Even though it was widely expected earlier in the summer.
It’s worth examining Williamson’s no doubt carefully chosen words in detail:
I would like the OfS to publish subject level TEF in 2021. This should be alongside the implementation of a new TEF model to be developed following the publication of the government response to the Dame Shirley Pearce’s Independent Review of TEF undertaken under Section 26 of HERA 2017. This new model should ensure the TEF is seamlessly integrated into OfS’s approach to the regulation of quality more broadly. To ensure that we do not lose momentum, I would also like the OfS to consider running a further provider-level TEF assessment exercise with results to be published in 2020.”
“Alongside the implementation of a new TEF model”
It’s the word “alongside” that is concerning me here. Subject TEF should be the implementation of a new TEF model in itself. You might recall the debates about the relationship between subject TEF and provider TEF and whether this should be top down (subject TEF expresses difference from the provider TEF rating) or bottom up (subject TEF ratings are combined, like mighty morphin’ power rangers, to produce provider TEF).
Both of these would have been a new TEF model. The method finally chosen – running the full provider TEF alongside a subject TEF that demands written statements for each of up to 35 subjects at the top level of the HECoS CAH, and then engineering some kind of read across by the main panel – also constitutes a new model. So what is going on?
The idea that TEF should seamlessly integrate into the OfS regulation of quality, should be (along with the modifications to the quality baseline suggested in another part of the letter) ringing alarm bells over at the Quality Assurance Agency. In designating the QAA as “designated assessment body” under the terms of HERA schedule 4, the OfS has vouchsafed that it:
would be appropriate for securing the effective assessment of the quality of, and the standards applied to, higher education provided by English higher education providers.”
TEF was initially framed as a quality enhancement tool, not a part of the quality assurance process. This incorporation of TEF ratings into quality assurance for the purposes of regulation feels like it would be an overreach by OfS.
A further provider-level TEF
Gods forbid that we lose momentum! Observers will recall how TEF1 snuck out under cover of darkness (it was just a list of all providers that held a current QAA review), TEF2 was a media firestorm, the excitement about which petered out for TEF3 and was almost non-existent for this years TEF4. A part of this has been due to the fact that TEF awards are designed to last for three years – only new providers, those replacing provisional awards and those who have seen metric improvements significant enough to move up a semi-precious metal level entered TEF3 and TEF4.
When the decision to bring subject and provider TEF together was taken, a delay was inserted in 2020 – and existing awards were extended for an extra year (lasting up to a maximum of four years for TEF2 holders) to compensate. The Glastonbury-esque fallow year was inserted for two reasons:
- To allow time both to finalise the combined mechanism and run what would need to be an elongated assessment process.
- Because HESA had discontinued the Destination of Leavers from Higher Education (DLHE) survey, and data from the elongated replacement (Graduate Outcomes) would not be available in time for 2020.
Both of these reasons still stand. There will be an extended lead in to the full subject plus provider 2021 run, unless there are enormous changes to the recommendations following the two pilots and consultation. And there just won’t be any suitable employment data to replace DLHE.
There has been talk about the use of the LEO data as a replacement – but this does not feature data on skilled/unskilled employment or on employment status on a particular date. OfS have always played fast and loose with award comparability – TEF2 awards are based on a weighted basket of metrics that is different from that used in TEF3 and TEF4. But it is difficult to see who benefits from an additional one year reweighting before a completely new system is brought in.
Certainly not prospective students. There is no evidence that students are using TEF in deciding which institution to apply to – OfS even tacitly admit this by omitting the ratings from the easy to find parts of Discover Uni.
4 responses to “What the (extra) TEF?”