REF 2028 is coming

If you’re a researcher your big question is whether or not you will be submitted to the next research excellence framework (REF).

And the answer, as of today, is no.

The awful idea of being “REFable” or not will have no salience next time round. There’ll be no requirement that any of your work is submitted – but the fact you exist and have research responsibilities counts towards the volume measure that determines how many outputs and case studies that can be submitted.

And of course, you’ll suspect that the quality of those outputs – as judged by your peers – are the single most important thing that determines your departmental REF score. And you’d be wrong – outputs will count for just 50 per cent of results, with measures of impact and the research environment taking up the remainder.

The story begins

Today sees the publication of the Future Research Assessment Programme (FRAP) initial decision report coded REF2028/01, a provisional equality impact assessment, and the international advisory report that has informed the development of both.

Assessment panels will be appointed at the back end of this calendar year, with the final guidance and criteria following in the winter of 2024 (following a consultation over the summer). You’ll be submitting to REF 2028 during 2027, with the all-important outcomes published in December 2028.

As a means of reducing burden many of the arrangements for REF 2021 will remain – feedback has suggested that adapting to new requirements is the principal driver of burden. So we’ll see the same panels, and some of the same definitions – and the REF rules for open access will duplicate the UKRI rules (though there is a consultation to follow on this).

But don’t let that make you think that REF 2028 is business as usual. A lot has changed.

Elemental forces

REF2028 will have, as REF2021 did, three elements, or profiles – though names and (notably) weighting are substantially different.

People, culture, and environment is worth 25 per cent of the overall REF score (up from 15 per cent for environment last time). This will be assessed on questionnaire responses collected at both institutional and disciplinary levels, supported by a “basket of indicators” (work on these will be commissioned later this year) drawing as far as possible on existing data. The new questionnaire template will be a lot more structured than previous iterations – there were concerns in REF 2021 that submissions here were not consistent, so this is an attempt to address this issue.

Engagement and impact, building on the REF 2021 impact profile, remains at 25 per cent of the overall score. The impact case studies – with the number required varying from one upwards based on the size of the unit of assessment in each provider – will be supported by an explanatory statement worth a minimum of 20 per cent of this profile, or at least one case study. The requirement that case studies need to be linked to research of at least 2* quality will be removed.

Contribution to knowledge and understanding will account for the final 50 per cent of the overall score. And there are big changes.

As indicated above, staff will no longer be submitted to the REF. Instead, the average full-time equivalent staff count, based on 2025-26 and 2026-27 HESA Staff returns will be used to determine the number of research outputs that can be submitted to each unit of assessment by each provider – 2.5 outputs per staff member contributing to a UoA.

This doesn’t mean that every staff member needs to have an output returned – indeed an institution can choose to return outputs from other staff members, including staff who do not have a specific responsibility for research and indeed anyone that can demonstrate a substantive link to the institution (defined as 0.2 FTE for at least 6 months).

There are no maximum (or minimum) number of outputs returnable for each member of staff. But an explanatory narrative statement (worth at least 10 per cent of the total marks for this element) will be needed to explain why the selected outputs are representative of that subject at that institution, and will include an evidence based statement about the institution’s wider contribution to knowledge in this area.

Research England has been very careful to emphasise that decisions on funding outcomes (currently based, in part, on volume) are separate to this process – and we should note a review of strategic institutional research funding is already underway.

Yes, but does it correlate?

As an illustration we’ve run the REF 2021 data using these weightings to calculate revised “overall” proportion (note that this is indicative only, doesn’t include the new approaches to assessment under each profile, and is limited by the resolution of the data available).

[Full screen]

The story here is not one of vast changes and recalibrating the REF entirely. It makes sense that excellent outputs have excellent impact and come from an excellent environment – so in the main this does not radically alter what we saw using the other weightings. There is an apparent correlation between excellent outputs, and excellent research environments, and excellent evidence of impact. And – notably – a very significant correlation between overall results calculated using the old and new weightings.

I built a tool to let you explore these relationships – you can set each axis on the chart to show the proportion of 4* activity in each UoA for outputs, impact, environment, the 2021 weighted overall assessment, and the 2028 weighted overall assessment (using the 2021 data, so not including the revised assessment plans).

[Full screen]

We appear to have hit a sweet spot between shifting weightings enough to drive institutional behavioural changes without fundamentally challenging the integrity of results. If the marginal gains energy of institutions becomes, as a result, directed less on the wilder shores of performance management and more on a supportive research environment we will all benefit.

Direction of travel

The International Advisory Group report, also published today, has clearly been (as it was commissioned to be) a huge influence on initial plans for REF 2028. This is a group charged with keeping an eye on international trends in the assessment and funding of research – and one of those key shifts has been away from the assessment of outputs to a systemic consideration of the “health” of research.

It’s from here we get the shift towards the institution and away from the individual, the recognition of the research work conducted by staff outside of the pool of traditional research active academics, and – notably – the reduction of weight placed on the assessment of outputs. Indeed, the IAG recommendation was that outputs should form only a third of the overall judgement – the reduction to a half (including the outputs statement) feels like a staging post towards this aim.

That the research sector is not sustainable, and is largely not livable, will not be news to anyone who works in it – and numerous UK government reports (notably Paul Nurse on the research landscape, Adam Tickell on research bureaucracy, and indeed the government itself in the people and culture strategy) have made this point. But this is an international issue – not one specific to the UK – and recent international initiatives (DORA, Leiden, the work of RoRI and Science Europe) exemplify what we may characterise as a de-escalation in research assessment.

The IAG report is unequivocal about the move away from the focus on the individual as a unit of research excellence, and towards REF being a tool to facilitate an integrated understanding of everything a research performing organisation does – research, teaching, and engagement – and the impact this has on staff. We can see the fruits of this in the incorporation of equality, diversity, and inclusion in the metrics and statements that will underpin the assessment of an institution’s (and UOA’s) strength in “people, culture, and the environment”.

Whither “excellence”?

There is very likely to be a push-back against the re-weighting of the REF by more traditionally-minded parts of the sector. The Research Excellence Framework, after all, is just that – so perhaps it should be primarily about the excellence of research that is produced?

What the FRAP project has done in rebalancing the exercises away from outputs is both brave and timely. Despite previous REF guidance actively encouraging the submission of novel forms of output, and the continued assertion that journal impact factors (and the idea of “prestige” in publication more generally) are both completely meaningless and irrelevant to REF assessment, for many the presumption remains that “excellent” research is a well-cited article in a “good” journal.

Research is a process not an end point, and the work of a huge group of staff in many roles rather than a single researcher. An output that can be reviewed by peers is one aspect of this (and it is encouraging that peer review remains central to the REF in a metricised world), the impact of research on the lives of us all is another, and the sustainability of every role that supports the ongoing capacity to conduct research is a third.

We may now have a research assessment system that fully recognises the process rather than just the outcomes. And this is something that anyone involved in research will surely welcome.