Impact can be defined as the difference research makes beyond academia, the tangible benefit arising from research. Taking this at face value, public engagement as an activity is a dissemination exercise, a route to impact or in Research Council terms a ‘pathway to impact’; it is not impact in itself.

Public engagement has always been and remains an important part of research, but not all public engagement will have impact that can be easily demonstrated, this does not make it any less important, disseminating and informing a wide audience whether within or beyond academia is of fundamental importance to research.

Has public engagement been devalued by REF?

In reality not all public engagement research is considered appropriate to become a REF Impact Case Study. This is partly because the tangible benefits to society arising from public engagement can be difficult to measure and evidence in a significant way. But then it has to be remembered that not all research is required to be submitted into REF.

Under the REF 2014 rules the number of case studies submitted to each Unit of Assessment was determined by the number of staff under that particular unit. For most institutions this required a relatively small number of impact case studies per unit, between 2 and 4 in many cases. Underpinning research projects with potential were assessed and reassessed by institutions and selected based on the perceived strength of evidence for impact, the reality in many cases favoured those projects that provided clear quantitative impact measures.

I was party to a webinar recently that included some REF Impact Assessor perspectives; one of the key points highlighted was a need to ‘distinguish activities, such as public engagement, from impact’ and included the advice that evidence should ‘enable assessors to distinguish activities from actual impact, e.g. spin-out companies and the sales or jobs they create’.

The concern that REF assessment panels may exhibit unconscious bias towards significant tangible metricised benefits over considerable public engagement outcomes is partly evident from these statements. I have witnessed similar attitudes in subject areas where metricised benefits are more commonplace, public engagement in these UoAs is easily side-lined as second-rate impact leading to institutional decision-making machines favouring clearer impact routes.

Will the Stern recommendations give public engagement more weight?

The recommendation in the Stern review to place more emphasis on public engagement activities in the next REF exercise has been welcomed by many, and rightly so. However, I question whether this will fundamentally change institutional practice in selecting impact case studies for submission. The potential for attitude bias to public engagement through an assessment exercise such as the REF may well remain in existence.

If this recommendation is implemented I suggest that there will need to be clear guidance both in terms of acceptable evidence for public engagement activities and in terms of assessment panel criteria for rating public engagement activities. This will give institutions the reassurance that public engagement case studies are worth investing in and will encourage debate and consideration as to how this range of impact generating activities are promoted and assessed.

Where does this leave Public Engagement for the next REF and beyond?

The value of public engagement is increasing, not only as a result of REF, but as part of the changing research landscape where the value of including and disseminating to our communities and the wider public are increasingly being recognised.

I have witnessed a growing change in attitude in traditional subject areas where public engagement would not be considered as a route to impact. It has long been recognised that public engagement activities are a beneficial addition to a pathways to impact plan, historically these statements have relied on standard institutional routes to public engagement in order to just ‘tick the box’ for impact. Units and individuals are now considering how to better incorporate public engagement into their project plans, looking for unusual and unique ways to disseminate to the public and encouraging cross-disciplinary partnerships between the hard sciences and the arts.

The arguments for change also work in reverse, those subjects reliant on public engagement need to look beyond their usual dissemination routes and consider other applications and potential collaborations for their research. New and interesting ways to robustly measure the benefits of engagement activities also needs consideration – is a survey really the only solution!

The impact agenda as a whole is giving public engagement a higher profile, encouraging activity and providing recognition to the hard work that goes into bringing research into the public domain; however, I would argue that there is still some way to go in changing mindsets.

Dee Summers is Research Impact Officer for the University of Portsmouth and writes here in a personal capacity. If you would like to talk about impact please contact me: dee.summers@port.ac.uk

An excellent view on the implications of the Stern recommendations for institutions is available here: https://researchandinnovationportsmouth.com/2016/08/03/stern-er-stuff/