The results are out and have been unpicked already with amazing speed and with interesting charts. And the criticism of TEF itself has inevitably intensified.
The problems with TEF are well documented – including the inadequacies of the metrics, the link to fees, the rumours and suspicion about links to visa quotas, and concerns about the Olympic-style awards system. Then there is the risk that, along with changes to the REF following the Stern Review, TEF will further separate teaching and research activity in universities. But, despite all these concerns, I believe that TEF is a good thing, and this first edition is a good step forward for the sector.
I say this having begun writing this blog before results were released, and expecting that my institution would be awarded Bronze. In the end, we are pleased to have been awarded Silver.
But whatever our award, it is hard to argue with the criteria that the TEF is evaluating. Surely no-one can argue that teaching quality, learning environment, and student outcomes are important and are things we should value. So it makes sense that we should measure and evaluate on them, in a peer review process that celebrates excellence wherever it is found.
Communication breakdown
Of course, unlike the REF, the TEF comes up with a single institutional award, with only three levels. It was not always the case that Bronze would be described as ‘good’; early discussions about the categories of award suggested Bronze would mean “needs improvement”. DfE promised that there would be clear communications correcting this, as indeed there was, and the HEFCE press release that accompanies the results gives some helpful context for the award definitions. But the early press coverage suggests that message has not been heeded, and that there is (perhaps inevitably) confusion about the difference between standards, as assured by QAA, and excellence as recognised by TEF.
But despite consultation, the sector failed to come up with a better naming structure for the awards. Our recent suggestion at Bournemouth was that two categories would be better than three. Poor communication or misunderstanding must not be allowed to damage the collective reputation of the UK’s universities at home or abroad, and there were mixed messages on that front from the minister on Thursday. Still, the medal classes can be reviewed later; the communication needs work now.
Opaque, but better
The inadequacies of TEF’s metrics have been covered extensively elsewhere. However, the written submission and the peer-review process has provided a counter-balance. There may be no metrics, or poor proxies, for many aspects of teaching quality, but there was an opportunity to address these in the written submission. So it isn’t right to say that the TEF doesn’t encompass such things – it just doesn’t use metrics to do it. The sector didn’t want a mechanistic, one-size-fits-all system that drove innovation out of education. We wanted an assessment that allowed different approaches to shine, and a peer review process to confer final judgement, as with REF. The results may be opaque, but they should be fairer and better as a result.
Which takes me on to benchmarking. Institutional benchmarking may make it tough to unpick the results, but it allows very different institutions to be compared on a fairer basis. It seems that the forced differentiation inherent in the benchmarking system has also been mitigated in practice by the submission and peer review process. Benchmarking will need to be kept under review because it may stop working over time. If the TEF drives improvement, then there will be less variance in institutions’ metrics, and the current benchmarking process will be less valid as results converge. But for now, it has helped.
Those opposed to the TEF argue that it won’t serve a supposed purpose: informing applicants. But as we said in our Green Paper response back in January 2016, evidence suggests that applicants don’t use the data available to them now. The current institutional ratings may not be particularly helpful to applicants interested in information about their own choice of course. But subject level TEF, if structured well, will provide better information to applicants and will drive the sort of improvements that the TEF was intended to provide. It will be a lot of work and may be uncomfortable, but it will be valuable.
Improvement can come through reputation alone
On the other hand, the link to fees has been completely unhelpful. It may have been politically expedient, but it has backfired. It has distracted attention from the main purpose of the TEF. Instead, there is suspicion and concern about the TEF amongst students, staff and institutions, which has fuelled concerns, such as links to visa allocation, despite all the assurances.
Institutions will work on the shortcomings highlighted by the TEF – we would have anyway, but the reputational impact of the TEF is enough to drive behaviour in itself. We don’t need differentiated fees to motivate us. Fee cap increases and differentiation linked to TEF awards not only makes the TEF toxic but assumes that the underlying market in fees is working. As has been written elsewhere on Wonkhe, there are many problems with a market-based view of higher education fees. In that context, the link to fees is unlikely to be effective. In fact, it has been postponed – let’s hope it doesn’t come back.
And finally, the TEF might encourage some institutions to drive specialisation for those who teach and those who research – we will have to wait and see – but it also allows universities to celebrate links between education and scholarship, research and practice. This is a choice for universities. At Bournemouth, we are choosing not to separate education and research – and we require staff to engage with practice as well. The risk of further division between education and research is not a reason to oppose the TEF (although it is a reason to push for changes to the REF).
As the Russell Group has said, this is a trial. It is a trial that will have consequences. But there is much that is good about it, including the criteria, the submission and peer assessment process, the data splits and the benchmarking (with caveats). As Chris Husbands has said, if we want better metrics we should suggest some. The current focus of the sector is on headlines and metrics – but a more qualitative analysis over the next weeks will be valuable. And the summer brings an opportunity to think about subject level TEF and how to make that work – I hope the sector will engage with that positively now the first shock of year two is over.
