Since the first REF 2021 draft guidance on submissions and panel criteria was published in July, the REF team have been on the road speaking to the sector and getting feedback on these key documents.
As the consultation period comes to an end, before we get down to business of analysing the written responses, we’d like to reflect on some of the main points we’ve heard during our travels.
First, the stats:
- 1,012 virtual attendees over five webinars
- 281 attendees over three HEI consultation events and two research-user workshops
- 63 REF team train journeys
- 348 lunches ordered (not all for the REF team!)
- 297 responses to our online survey.
Codes of practice
One of the specific areas on which we were seeking feedback was how we can best take account of the different personal circumstances that may have affected researchers during the REF period of assessment. There has been plenty of discussion on the proposals we have outlined, from which it was immediately clear that there is wide consensus on the importance of ensuring the exercise supports equality and diversity in research careers.
There has been greater divergence in views, however, on how this can best be achieved within the new assessment framework. Many have voiced support for the clarity and consistency offered by defined reductions in the number of submitted outputs required by the unit overall, on the basis of individual researchers’ circumstances. Others have raised concerns about how such reductions can meaningfully be passed on to affected staff in a way that will ensure achievement of the intended aim.
Related to this issue is the development of HEIs’ codes of practice, which are intended to promote fairness and transparency in the processes followed in submitting to the REF. There has been a lot of engagement about putting a code of practice together and discussion about whether the June 2019 deadline for submitting codes would be too late for some and too soon for others (whether this suggests it’s about right remains to be seen).
As part of the discussion on codes of practice, there has been some reflection on HEIs’ processes for identifying which staff have significant responsibility for research. We’ve heard some examples of good practice in this area, but also some remaining uncertainty and a general desire to have the opportunity to explore emerging processes in more depth before the code of practice deadline.
Determining eligibility
Given the changes to staff and output eligibility in REF 2021, it’s unsurprising that these areas have generated a lot of discussion. We invited comments on our proposal to make ineligible the outputs of former staff who have been made redundant (except where voluntary). Many have voiced support for this policy but we’ve also heard a number of concerns that the exclusion of these outputs may not, in fact, be desirable for the individuals affected, particularly where they are early career researchers.
We also sought views on whether staff who are employed by, and whose research is focused in, the submitting HEI but who are based in discrete units overseas should be eligible for submission. We’ve heard clear views that this would have a significant effect on certain disciplines (e.g. tropical medicine) and we are working with institutions to explore this further.
Another clear issue that’s been raised is on the question of how to take research independence into account for those on teaching and research contracts, which has emerged as perhaps a wider issue than has been indicated in our earlier consultations.
Coloured boxes
We were pleased to discover widespread support for the combined panel criteria and particularly delighted that the coloured boxes – the subject of lengthy discussion – have gone down well. The panels worked hard to achieve consistency across their criteria, ironing out differences where possible, and retaining disciplinary nuances where necessary.
But there is appetite for even greater consistency, particularly in the panels’ advice on impact. It will be a key task for the panels now to weigh up the full set of feedback and agree where the boxes should remain and where black and white should replace colour.
Finally, we are grateful to all those who have pointed out where the guidance is unclear. After many months working through twenty plus versions of the documents, we really needed a fresh pair of eyes.
The next steps now for the REF team are to analyse and digest the consultation responses with the panels at their meetings in November and December, and agree the final decisions. Then a busy Christmas will see us prepare the final documents for publication in early 2019.
