How the REF is used determines how the burden is felt

Government announced a fundamental review of the future of research assessment on 19 May 2021. For many of us, this will feel like Groundhog Day.

However, although the latest Research Excellence Framework has delivered substantive and welcome changes, there remains work to be done. This year’s Easter break just after the REF submission deadline was extremely welcome to all colleagues involved.

So, how do we use research assessment without the high burden it creates for academic and professional colleagues in universities? And, with so much attention on our research culture in the UK (and beyond, how can national research assessment play a more constructive role?

I have spent a substantive part of my career on the funder side, as well as in a number of UK universities. I have observed personally how usually well intentioned national policies can lead to unexpected (and undesirable) consequences within universities, including some policies which I myself helped to design. In 2018-19, I had the opportunity to undertake a Master’s research project and to interview more than 30 participants representing national funders, university management and the academic community about these questions and the REF. So, what may help us in collectively designing a better and more efficient approach to research assessment in the UK?

Efficient for everyone

The way universities use the REF is key here, and its local impact may be underestimated by national policy makers. As one national funder has noted:

The REF is not vast enough or specific enough to be really used as a real-time management tool

However, rightly or wrongly, the reality is that universities have in general used REF to bolster their own management of research performance.

The highly structured and detailed requirements for REF submissions provide a clear sector-wide framework. This makes it easy to roll-out locally and to defend the approach and the aspects of “research performance” that are being measured. University research performance reviews have though given rise to a new local industry of performance management, in part explaining the perceived burden of the REF.

Provider usage

The REF is just too attractive an instrument for university performance management to ignore. Any changes to our national research assessment framework are unlikely to remove the requirement and desire of universities to manage performance locally. Therefore any call to reduce the burden of research assessment is likely to fail if it does not also address the likely adoption of any national framework within universities. One senior academic expressed it:

REF has given university senior management a powerful tool in terms of developing university performance reviews. These drive the system perhaps even more than REF itself.

Universities have primarily used the REF framework at departmental or unit of assessment level, where REF appears to have filled a gap in management capabilities and provided a certain level of transparency. Explicit institutional use of REF to manage individual research performance is an exception. As one senior university manager said:

REF is a management tool that should be unnoticed by the individual academic.

However, in the subsequent trickle down from departmental level, REF starts to shape expectations of individual performance as well, if less explicitly and often not intentionally. This is a cause of concern for many academics: “people can feel very threatened by REF and being judged”. Concerns include the divisiveness of local peer review; short term focus of REF when contrasted with long term career objectives; and a perception that individuals have to deliver to all of the REF performance criteria at the same time – four star outputs and four star impact case studies.

Departmental dynamics

If we wish to use research assessment as one tool to change our research cultures, then we need to understand and influence the dynamics and consequences of research assessment at departmental levels. The relationship between senior, university-wide and middle, departmental level management is highly influential in how interventions are perceived by the academic community.

So, when we as a sector discuss a new research assessment framework, we should take these wider system impacts into account. REF is highly mediated by universities. Universities should recognise their critical role as translators, but universities are complex organisations. The consultation process should ensure views beyond those of senior university managers are taken into account. Actively reaching out to departmental academic managers would be a good starting point to get a more rounded insight and understanding local impacts.