TEF will succeed if it leads to enhanced learning for ALL students

I welcome this consultation on the future direction of the Teaching Excellence Framework (TEF). It may have taken a while but it’s worth waiting for.

The Office for Students (OfS), supported by the TEF metrics and TEF advisory groups, has considered the findings of the independent review of TEF which I chaired, and which was published in January 2021, alongside the Department for Education’s response, and there is much in OfS’ proposals that I think is great.

It’s alive!

Firstly, it’s great that TEF is alive and well. For all its limitations, TEF has had a powerfully positive impact on UK HE. It shone a light, in all institutions, on the quality of the educational experience and led to the needs of students rising up the agenda for senior leaders and governing bodies.

And it’s great that our primary recommendation about clarity of purpose has been responded to so seriously. In the past there has been ambiguity about whether the purpose of the TEF is primarily to provide information for students or a lever to improve quality of provision. Enhancement of the provision of HE is at the core of these proposals. It influences the proposed process, as well as the data used, to inform the ratings.

It is, however, a bit surprising that the informational value of the ratings is still discussed as much as it is, suggesting that the idea that TEF can act as a source of public information on quality has not gone away. While TEF can offer a steer, an institutional measure can only tell a student or employer so much, and so in making decisions about the final TEF process, the enhancement of provision for ALL students should be prioritised over information provision. This is particularly important given that we have a regulatory regime where the levers for enhancing the quality of provision are scarce but sources of information are multiple.

On this theme I also welcome acceptance of our recommendation for a formal student submission as part of the assessment process. I know this will be a challenge for smaller student bodies but it’s clear that OfS recognises this and will work to accommodate it.

Diversity and data

I emphasise ALL students because one of the key messages from the listening sessions, and call for views that we held as part of the independent review, was the diversity of needs of our students, and the diversity of mission, methods, and resources of our providers.

It was recognition of this diversity that led to our recommendations about changes to the process and altering the balance of qualitative and quantitative data in making overall ratings.

So, it’s good to see proposals which determine ratings of excellence from an equal weighting of institutionally derived and nationally comparable data. This is essential if provision is to be assessed in context, particularly at institutions that teach students who start their course with greater learning needs. And it’s also great to see educational gain in there as part of the institutionally determined data – an opportunity to contribute to the evidence on educational gain in a nuanced way that I hope institutions will take up.

The proposed process also takes on board the tremendous work of ONS who investigated our concerns about both statistical and methodological weaknesses of the previous TEF. The previous approach to the TEF used a statistically driven initial hypothesis as the starting position from which to judge the quality of provision. If the initial hypotheses were based on weaknesses in the data or statistics, this then made it harder for institutions to prove their worth.

This new, more balanced, approach will remove the predictive power of the statistically driven initial hypothesis making it easier for institutions to demonstrate what their excellence looks like and to have this count towards the overall rating in a more proportionate way.

It’s great to see a core place for NSS as one of the nationally comparable metrics. NSS is an important measure of what students think about their experience – but it’s also reassuring to see an end to the use of NSS as a pseudo-objective measure of educational quality.

And it’s great that OfS has taken on board the need for greater transparency in the way data is used. The tables showing the ranges and implications of the ranges in quantitative data are to be very much welcomed. And the polarisation of views around the importance of either relative or absolute comparisons of date is addressed (in Para 184) in a clear and transparent way.

Subject review

Coming back to ALL students. We concluded in the review that subject level TEF failed to meet the requirements for robustness and reliability and might undermine the credibility of the provider level ratings. We therefore recommended that subject level ratings should not go forward.

But we heard, from all parts of the sector, that the data informing the subject level process were powerful, within institutions, in facilitating discussion about enhancement of provision. It raised senior leaders’ awareness of areas of their provision that were not delivering for ALL students. In other words, it had a direct enhancement function.

We therefore proposed a subject level exercise to assess the provider’s performance in delivering enhancement and a recommendation that this judgement should inform the overall provider rating.

OfS has built on the review proposals and come up with a process that makes annual subject level data, with splits, available to institutions. The regulator will also assess the provider’s response to this as part of the four year cycle of provider assessments.

This should be implemented in a way that keeps the needs of ALL students firmly in mind. Overall institutional excellence at provider level must not be able to obscure even small areas of poor performance or the neglect the needs of any group of students. I hope the term “ALL” will be introduced in the “features of excellence” descriptors and other parts of the documentation.

This availability of data and information about all splits, not just subject level, will mean institutions will need the capacity to analyse and understand the data if they are to make best use of it. I hope institutions will continue to see this as a significant facilitator of enhancement and not construe this as a burden.

Potholes

There is a lot that I think is great about these proposals. But there are some missed opportunities and some potholes too.

I think it’s a missed opportunity to continue to call this a teaching excellence exercise. We know that what HE provides is a complex all round set of learning experiences which shape resilience, team working, and leadership, to name but a few of the qualities important for future success.

These skills and experience are learned from the wide range of opportunities that HE provides. It’s not just about the teaching, as has been clear during the pandemic. And it’s not just cosmetic. The name frames expectations about what is being measured and what is important. That is why we recommended the name Educational Excellence Framework.

We also proposed more relevant terms than Gold, Silver, and Bronze, which we heard are confusing for international audiences and where Bronze had come to be seen as a failure rather than a level of excellence, but which are retained in the OfS proposals.

A more substantive concern relates to the proposed name of the lowest rating. “Needs improvement” raises a lot of questions which I anticipate will cause problems. TEF is a measure of excellence above a baseline. What does “needs improvement” mean?

Without a clear indication of what needs to change and how that relates to registration requirements, it brings risks. Whilst the regulator can legitimately demand baseline requirements to be met for registration this is different from “excellence” which cannot reasonably be required as part of a baseline requirement.

In the review we proposed that if no examples of excellence can be identified, the pragmatic and accurate term for the bottom rating should be “meets baseline requirements.” If baseline requirements themselves are found not to be met this should trigger a separate investigation with possible suspension of registration.

Whatever we put in place at the start of the new TEF will evolve over time. I hope that any future changes will follow the three principles we proposed: transparency, robustness, and relevance to purpose (enhancement).

We heard that previous changes had not always been understood, or well explained, which led to concerns about its validity. The overall success of the exercise is dependent on the sector having confidence in its purpose and process. Changes should only be made if they help improve the core purpose of the exercise. The enhancement of the learning environment for our students is too important to be knocked off course by political interventions or administrative convenience.

I congratulate all involved in getting to this point and hope that there will a wide ranging response to the consultation. The power of the TEF to promote excellence in UK higher education is critically important to us all. Our students, to whom we entrust our futures, deserve the greatest educational experience that we can provide.