Regardless of where you stand on the TEF, there is no denying that it has been a catalyst for fervent debate since its launch in 2016. That it has attracted a lot of media attention and continues to be a topic of conversation on campus is not only testament to the widespread interest in the quality of HE teaching per se, but also the depth of feelings the TEF has provoked as a piece of policy reform.
From the National Union of Students, to TEF policy wonks, university vice chancellors and Universities UK, there has been no shortage of contributions to the debate by HE stakeholder groups. But one noticeable group of voices has been conspicuous by its absence in the development, implementation and assessment of the TEF to date – HE teaching staff.
Today’s publication of our research report “Understanding, recognising and rewarding teaching quality in higher education: an exploration of the impact and implications of the Teaching Excellence Framework” therefore marks a key milestone in the TEF policy debate, as it captures the voices of some of those most directly affected by this policy reform. The report includes the views and experiences of over 6,000 academic staff working in universities and college-based HE providers in England, Scotland, Wales and Northern Ireland.
Commissioned by the University and College Union (UCU) to provide an evidence base of the impact and implications of the TEF on their members working in university/college-based HE, myself and colleagues at Birmingham City University undertook the research from February to November 2018. The study investigated UCU members’ awareness, involvement and perceptions of the TEF and its impact on their professional lives in their workplaces to date.
An unpopular policy
So, what did we find out and how might our findings inform the Independent TEF Review?
Our research reveals that the TEF has proven to be a highly unpopular policy with the vast majority of the project’s participants, with only one in ten welcoming its introduction. It calls into question the fitness for purpose of the TEF as a vehicle for making valid and reliable judgements about teaching excellence and its ability to fulfil its purported aims of improving the quality of teaching and increasing student choice in HE.
Participants’ perceptions of the conceptualisation and methodology used to capture teaching excellence in the current TEF was that it failed to do so in any meaningful way. Not only is the TEF not a direct measure of the quality of teaching, as acknowledged openly by the chair of the TEF assessment panel, Chris Husbands, when interviewed for the study, but it was not considered a credible measure at all by the overwhelming majority of the study’s participants, as emphasised repeatedly across data sets. This is quite a damning indictment of a policy that, according to Husbands, is unlikely to disappear in the near future:
I’ve got no reason to suppose that the TEF is going to disappear … And I suspect, although I’ve not spoken to anybody in the opposition, that it’s too useful a tool for any incoming government to want to get rid of it. So I think this is part of the landscape and we’ve got to make it as effective a part of the landscape as it can possibly be.
Husband’s statement inevitably raises a number of questions about the future of the TEF. How and what is the TEF useful for? For whom? Beyond its use as a market signal for categorising HE providers according to its ranking system, who actually benefits from the TEF? And how can it become (according to Michael Barber) “catalyst for the improvement and innovation in the quality of teaching” when it is based on a unidirectional, episodic, desk-based assessment of a collection of data sets, much of which have only tenuous links to teaching and teaching excellence?
During a research interview I carried out as part of the project, Chris Husbands repeatedly used the analogy of the TEF assessment processes as a vehicle for ‘telling a story about the institution’. But who are the narrators of this story? Who is the audience? And to what extent do the protagonists of this story have the opportunity for their voices to be heard?
The wider question of teaching quality
We see a pressing need for a national debate on what teaching excellence actually is. Any debate should ensure the representation of all relevant stakeholder groups, notably policy makers (e.g. the OfS), the HE workforce, students, student bodies (such as the National Union of Students), senior leaders and professional bodies representing the sector.
Any framework or approach aimed at improving teaching must include staff at all levels. Institutions need to ensure that the consultation, creation and communication of all TEF-related (or any other national teaching policy) work (from policy formation to implementation) include a broader representation of all staff and not just an elite group of those in (senior) management and/or professional services. This requires a more inclusive approach to involving staff in shaping institutional policies and practice.
Despite the flaws and criticisms of the TEF expressed by participants in our study, understanding, recognising and rewarding teaching excellence was repeatedly acknowledged as an important mission, and broadly welcomed by the HE workforce. Staff were very committed to the idea of improving teaching quality and were keen to share and collaborate on ideas and practices.
Lessons from other sectors
Across the study, participants provided a wealth of ideas and suggestions for alternative, more representative ways of considering, talking about and capturing teaching excellence. They also had a lot to say about alternative methodologies, approaches and/or considerations for understanding, improving, recognising and/or rewarding teaching. Such was the wealth of ideas generated by participants that a separate section of ‘Alternative visions of teaching excellence’ was created in the report.
There are valuable lessons for HE to learn from the schools’ improvement agenda about the counterproductive effects of relying too heavily on the use of metrics to assess and performance manage educational provision. One of these lessons is that there needs to be less focus on showcasing manufactured manifestations of excellence in response to accountability-led regimes and more on supporting collaborative and collegial work across institutions that is responsibility-led.
Whilst recognition and reward schemes may act as short-term incentives, there is little evidence that they lead to long-term, sustainable improvement either individually or institutionally. This is something that comes from supporting not sorting staff. Whether the TEF or a variation of it remains part of the future HE landscape or not, our research reinforces the need for a radical policy review that places greater emphasis on supporting HE providers to collaborate rather than compete with each other to improve the quality of teaching.
We need to see the REF in the context of the entire system. The REF distorted priorities in the University sector to the production of research, much of which consists of obscure articles in journals no one reads other than other academics in the field. Indeed, that is why the concept of Impact had to be introduced. Teaching became a “regrettable necessity” and a diversion from the production of research papers. Notwithstanding its many failings, the TEF did achieve a rebalancing of attention to education. The fact that many illustrious research institutions received lousy TEF ratings was an acknowledgment of what “everyone knows” – they get good outcomes in things like DELHE not because of the value that they add in instruction but because of their highly selective intakes. Which brings me to what I see as a prime failing in the current TEF – the failure to focus on “value added”, how much further along students come given what one would predict given their family, location and school background.
As to dissatisfaction of University staff, so long as hiring and promotion decisions are primarily driven by research contributions, we can expect such tension. We need to be clear about who Universities are supposed to benefit: students and the community, or their staff. The University world has for long been in a condition of constant growth, where such difficult questions could be avoided by further expansion using an increasing inflow of money. Now that this has stopped, it is time to reconsider priorities.