Impact beyond the REF. Consider that for a moment. What does that mean? What would that look like? And with REF so often synonymised with impact in the UK, how can we extract ourselves from its dominant voice?

I was invited to deliver the opening plenary – alongside Melanie Knetsch (ESRC’s Strategic Lead for Interdisciplinarity, Innovation and Impact) and Professor Peter Kemp (Vice-Dean for Academic Affairs and Professor of Public Policy, Blavatnik School of Government, and University of Oxford Social Sciences Division Strategic Impact Lead) – to consider impact beyond the REF. It’s an odd concept because – let’s face it – REF isn’t going anywhere, so establishing ‘beyond’ is immediately difficult in tone.

Ultimately there are numerous ways to look at this.  As my plenary colleagues considered the bigger strategic picture and the policy context, I chose to think about impact on-the-ground and in the lives of researchers, research managers and ‘users’ (more on that later). Here are my headlines:

First things first, impact existed before the REF.  I hope you were sitting down

I don’t want to startle you, but REF didn’t invent impact. I know – it’s a shock. Of course we had impact before REF, we just didn’t really call it that. I started my research career in 2003 and anything beyond the academic world was generally captured by words like ‘applied’, ‘knowledge transfer’ or in my own discipline ‘(health psychology) ‘implementation’.  Impact (with a big ‘I’) didn’t really hit our radar until its introduction to REF in 2011, and even then, only those involved in a REF 2014 submission had to invest effort in tracking, evidence gathering and causative narration. But impact is neither exclusive to or borne from REF. Please breathe again.

REF is a double edged sword (or ‘let’s elevate the value of applied work…..then make them dance for money’)

Unlike previous research assessment exercises, REF elevated the importance of research-driven real world change. For many of us, that recognition offered a welcome rebalancing of what it meant to ‘be an academic’.  However, formalising impact within assessment agendas also introduced weighting, ranking and a financially incentivised need to drive bigger, stronger, more important and further-reaching effects.  Whilst this has triggered investment and innovation, we’ve also witnessed a culture of anxiety and substantive institutional restructuring. If we can continue to use REF to catapult creativity, balanced with fair expectations (and reduced worry about repercussions of not doing impact), perhaps we can really deliver social benefit.

A need for impact literacy across the research system. 

Impact is important, but not all research should have impact. We need discovery science, just as we need philosophical shake-ups of academic assumptions. But we also have a duty of care to pass it forward, whether that’s beyond the academic wall, or to other, perhaps more applied academic counterparts. Everything we know about impact says it can’t be templated and is achieved through multiple routes. Some are scripted, some are opportunistic, but all involve decisions about the best and most purposeful approaches. With so many possible permutations, it’s incumbent on the research community to become impact literate (paper here or OA version here). Let’s be clear,  I don’t mean ‘more training for academics’. Of course training is needed, but what I’m talking about is an understanding of how impact ‘works’ across academics, research managers, institutional leadership, funders, publishers……..a commitment to recognising that impact looks, feels, and operates differently across disciplines and cannot be reduced to metrics at the ‘end’ of the journey.  Joyfully we are seeing considerable energy in this direction – Responsible Metrics calling out bad practice in quantifying effects, drives towards impact literacy embraced by publishers, ‘knowledge mobilisation’ arguably overtaking ‘transfer’ and ‘exchange’ as more expressive terms of implementation. We still have a long way to go, but the more everyone ‘in the frame’ is equipped to judge how their work connects into impact, the fairer (and more effective) we will be.

Finding meaning with ‘users’, not just ‘transferring knowledge’

I personally dislike the word ‘users’, as it conjures up images of some transactional process whereby academics apply their research to a waiting horde and *TA-DA* the public cry tears of joy that their lives are better. I’m not sure quite what language is better (very happy to be advised!) but whatever we use, we need to shift away from simplistic views of impact-as-productisation and impact-as-consumption. Impact can and must always be measured at the level of the beneficiary/ies, in terms of what is meaningful for them.  As we move forward, we’re seeing co-production and patient and public involvement weaved more substantially into research delivery, and this shift of focus away from blunt assessment currency can only be a good thing.

Disrupting simplistic, linear models of ‘ultimate’ impact

Logic models of impact are great (and endorsed fabulously elsewhere) but they can unintentionally reinforce an expectation of linearity. Input-output-outcome models routinely position impact as the ultimate endpoint of a research journey. Fine. Well not fine, actually, if that shifts assessment and thinking to so far down the line that it blinds us to those meaningful effects along the way.  I’ve written before about the dangers of focusing only on ethereal and magical endpoints but if we can truly understand the iterative and co-produced nature of impact, we can make some very meaningful changes indeed.

Valuing and investing in impact professional development or ‘There are people that do this stuff? Like, willingly?’

I’m extremely proud to lead the ARMA Impact Special Interest Group, a group of over 750 research managers with a core, or connected focus on impact. This community of practice grows every year in both size and combined expertise, and offers the most incredible combined set of insights into what it’s like to ‘do impact’. Yet REF risks painting a simple one to one picture of academic (project) to impact, leaving brokers and research manager invisible in impact discourse. Impact needs a strong and valued community of practice, both in terms of dedicated impact professionals and the integration of impact into academic career development. If we don’t acknowledge the skills, efforts and expertise inherent to driving research into practice, quite how much impact can we really expect to have?

Impact does not, and never has equalled REF. Impact is not about assessment compliance, nor is it about demonstrating effects at scale at the expense of meaningful change of any size. We need to recognise impact may be driven by REF, elevated by REF, or conversely deprioritised if it doesn’t ‘fit the REF mould’.

Impact already exists beyond REF.

So perhaps the question shouldn’t be what impact looks like beyond REF, it’s what you’d do even if it wasn’t being measured.

*This blog accompanies a plenary at the Making a Difference: Impact in the Social Sciences’ Conference, April 19th 2018, St Anne’s College, Oxford*