As part of our preparations for REF 2021, the University undertook a REF readiness audit in autumn 2018.
Different departments, schools and faculties were at different stages of preparedness, as you would expect, but early signs show that we expect to submit considerably more staff and outputs than in REF 2014.
Unlike in 2014, we will not be choosing which staff are submitted to REF in 2021. Instead we must use a fair and transparent process to assess which academic staff hold “significant responsibility for research”.
Those submitted will go into one of the university’s chosen Units of Assessment, aligned to the disciplinary area of their research. Each person must submit at least one output, and may submit up to a maximum of five. Units must have a total number of outputs equal to 2.5 per person (FTE). Outputs are worth 60% of the total REF result.
Each Unit must also provide a statement about its research environment, worth 15%, and must demonstrate practical outcomes from its research through Impact Case Studies, worth 25% of the result.
Every Unit must have at least two Impact Case Studies, with an additional case study required for each 15 staff (FTE) above the first 20 people.
The audit was designed to look primarily at staff and outputs. To mirror the real REF in 2021, a “census date” of 31 July 2018 was used to assess whether staff met the requirements for inclusion in a REF submission. Outputs were then proposed and assessed, providing a “quality profile” for each Unit and for the University as a whole.
The audit was also an opportunity to test our REF processes and policies, some of which have now been improved or updated. It does not therefore give a perfect picture of exactly who will and will not be submitted to REF in 2021, nor does it give our exact expected quality profile of outputs. What it does give is an idea of how we are doing so far, and where we need to do more work to ensure we are ready for our real REF submission in 2020.
It was very encouraging to see both the number of staff likely to be submitted to REF in 2021 and the proportion of eligible staff who were submitted increase considerably on our REF 2014 numbers. Part of this change is due to the changes made to the REF exercise, moving it from being a selective exercise to a more inclusive one. But it is also a reflection of increased engagement with research across the University.
Even more positively, the quality of the outputs assessed in the audit also increased, with a higher proportion receiving 3* or 4* ratings.
A Grade Point Average, or GPA, was also calculated for each Unit, and overall for the institution. GPAs give a single quality score by combining the proportion of 4*, 3*, 2*, 1* and unclassified outputs a Unit or institution has achieved.
These audit GPAs were just for the outputs, so unlike the final REF GPA they do not include the environment and impact elements which, as noted earlier, make up 40% of the final score.
Our GPA for the audit was 2.65 – only 0.02 lower than our output GPA for REF 2014, and a very positive outcome given the number of unclassified outputs we had to include for administrative reasons. If those unclassified outputs are discounted, the overall GPA becomes 2.97. As many of the issues that caused the problem with the unclassified outputs have now been resolved, this is closer to what our “real” GPA would currently be.
If you’d like to know more about how the audit was conducted and the detailed results [UoP Staff only], get in touch with your Unit of Assessment Coordinator or the Associate Dean (Research) for your Faculty, who will be able to provide you with a copy of the report that went to University Executive Board.
Although the audit focused on outputs, it did also look at the readiness of our current long- and short-listed impact case studies. For the audit, a number of possible case studies were considered, with most already able to demonstrate narrative, underpinning research, and evidence.
Impact is an increasingly important area of REF, as the move from 20% to 25% of the final score coming from this element shows. It is ever more vital that we not only ensure we have impact case studies now, for submission to REF 2021, but also that impact, and gathering of evidence of that impact, is built into current and future research.
With this in mind, our newly-appointed Pro Vice-Chancellor (Research and Innovation), Professor Bob Nichol, recently launched an Impact Acceleration Fund. This fund is specifically targeting our existing, short-listed case studies and aims to help them improve and develop with a view to securing the highest assessment possible.
If you want more information about applying for this Fund [UoP Staff only, please], get in touch with your Faculty Impact Lead or your Associate Dean (Research), or contact RIS at firstname.lastname@example.org.
Next steps are already underway as the REF Team in RIS work with REF Steering Committee, Associate Deans (Research), the Impact Working Group, and others across the university to make sure we learn the lessons from this audit and make any changes needed.
Our thanks to all colleagues across the university who took part in this audit exercise. We know it was a busy time and another thing to do on top of the day job, but a lot of incredibly useful and important things were learned that will help us achieve the best REF 2021 result we can.