One of the more controversial aspects of the recent REF consultation was the proposal to use HESA contract data to determine which staff are ‘research active’ and therefore returnable on a compulsory basis to the next REF.
The recommendation provoked a strong response from the sector but two warnings in particular stood out. First, since not everyone with research in their contract is required to produce ‘REFable’ outputs, institutions would be left in the position of either including such individuals in their submission and taking a hit to their quality profile, or switching staff onto teaching-only contracts at the expense of people’s careers. The units most at risk would be those in a heavy practice-based subject such as allied health, education, architecture and law. These are populated by individuals whose expertise and authority to teach is informed by something other (but no less valuable to the ecosystem) than research leading to REF outputs.
Second, in some circumstances, the proposal to use HESA data would result in REF panels assessing a high volume of lower scoring outputs which institutions did not want to submit. This would obfuscate the picture of ‘research excellence’ which the REF is supposed to give us.
So came a collective sigh of relief when two days before the consultation closed HEFCE’s David Sweeney acknowledged that contracts do not necessarily reflect a person’s work pattern and an alternative solution may be in order.
The case for a ‘significance test’
HEFCE’s consultation was of course based upon the Stern Review, which recommended that “all staff with significant responsibility to undertake research” should be returned to the REF. This would move us away from the highly selective process seen in REF 2014, towards the universal submission of researchers. But from an institutional perspective, there may be a good argument for retaining the ‘best-foot-forward’ approach. Full selectivity not only enabled institutions to pinpoint pockets of research excellence wherever they exist, but also build capacity in those areas for enhancement.
It seems the current mood, however, is to honour the Stern principle that any staff member with “significant responsibility to undertake research” should be included in the REF. That being the case, University Alliance believes that institutions should be responsible for identifying those individuals and presenting the criteria and data used in selection for audit on the REF census date.
A test of someone’s significance as a researcher should be underpinned by a Code of Practice and be amenable to audit through random sampling and other methods. REF 2014 demonstrated that a Code which is fair, balanced and transparent, though not conditional on individual staff consent, is workable, and therefore repeatable.
Outputs
A revised model for determining which individuals should be returned to the REF requires us to revisit the number of outputs per researcher. The Stern Review suggested flexibility in this regard, offering an average of two per head across each unit of assessment with a maximum of six and a minimum of nought per staff member.
Subject to the adoption of our significance test however, there may be a case for stipulating a minimum of one output with the expectation that someone with significant research status will have produced at least a single publication within the seven-year REF cycle. In this scenario, acknowledgment would need to be given to special circumstances such as parental leave, career breaks, or fractional periods of service.
Importantly, selection in the last REF proved to be time consuming and controversial because of the impact on the career of someone omitted. Reducing the minimum threshold of outputs to one would mitigate both these problems.
A lower limit on outputs could be preferable too. Under a maximum of six, ‘superstar’ researchers would be disproportionately represented in the research picture. This would produce a less indicative view of the landscape as a whole. A lower limit of four outputs would give less weighting to ‘superstars’ and result in the inclusion of outputs from a broader base of researchers, fulfilling another of Lord Stern’s intentions.
In summary…
The significance test captures the spirit of the Stern Review but has none of the downsides of the HESA proposal. In the interests of the research base, the government must be persuaded to adopt it or we will end up down a road that nobody – not least Lord Stern – ever intended to go.
