Making a difference in impact….is there life beyond REF?

Impact beyond the REF. Consider that for a moment. What does that mean? What would that look like? And with REF so often synonymised with impact in the UK, how can we extract ourselves from its dominant voice?

I was invited to deliver the opening plenary – alongside Melanie Knetsch (ESRC’s Strategic Lead for Interdisciplinarity, Innovation and Impact) and Professor Peter Kemp (Vice-Dean for Academic Affairs and Professor of Public Policy, Blavatnik School of Government, and University of Oxford Social Sciences Division Strategic Impact Lead) – to consider impact beyond the REF. It’s an odd concept because – let’s face it – REF isn’t going anywhere, so establishing ‘beyond’ is immediately difficult in tone.

Ultimately there are numerous ways to look at this.  As my plenary colleagues considered the bigger strategic picture and the policy context, I chose to think about impact on-the-ground and in the lives of researchers, research managers and ‘users’ (more on that later). Here are my headlines:

First things first, impact existed before the REF.  I hope you were sitting down

I don’t want to startle you, but REF didn’t invent impact. I know – it’s a shock. Of course we had impact before REF, we just didn’t really call it that. I started my research career in 2003 and anything beyond the academic world was generally captured by words like ‘applied’, ‘knowledge transfer’ or in my own discipline ‘(health psychology) ‘implementation’.  Impact (with a big ‘I’) didn’t really hit our radar until its introduction to REF in 2011, and even then, only those involved in a REF 2014 submission had to invest effort in tracking, evidence gathering and causative narration. But impact is neither exclusive to or borne from REF. Please breathe again.

REF is a double edged sword (or ‘let’s elevate the value of applied work…..then make them dance for money’)

Unlike previous research assessment exercises, REF elevated the importance of research-driven real world change. For many of us, that recognition offered a welcome rebalancing of what it meant to ‘be an academic’.  However, formalising impact within assessment agendas also introduced weighting, ranking and a financially incentivised need to drive bigger, stronger, more important and further-reaching effects.  Whilst this has triggered investment and innovation, we’ve also witnessed a culture of anxiety and substantive institutional restructuring. If we can continue to use REF to catapult creativity, balanced with fair expectations (and reduced worry about repercussions of not doing impact), perhaps we can really deliver social benefit.

A need for impact literacy across the research system. 

Impact is important, but not all research should have impact. We need discovery science, just as we need philosophical shake-ups of academic assumptions. But we also have a duty of care to pass it forward, whether that’s beyond the academic wall, or to other, perhaps more applied academic counterparts. Everything we know about impact says it can’t be templated and is achieved through multiple routes. Some are scripted, some are opportunistic, but all involve decisions about the best and most purposeful approaches. With so many possible permutations, it’s incumbent on the research community to become impact literate (paper here or OA version here). Let’s be clear,  I don’t mean ‘more training for academics’. Of course training is needed, but what I’m talking about is an understanding of how impact ‘works’ across academics, research managers, institutional leadership, funders, publishers……..a commitment to recognising that impact looks, feels, and operates differently across disciplines and cannot be reduced to metrics at the ‘end’ of the journey.  Joyfully we are seeing considerable energy in this direction – Responsible Metrics calling out bad practice in quantifying effects, drives towards impact literacy embraced by publishers, ‘knowledge mobilisation’ arguably overtaking ‘transfer’ and ‘exchange’ as more expressive terms of implementation. We still have a long way to go, but the more everyone ‘in the frame’ is equipped to judge how their work connects into impact, the fairer (and more effective) we will be.

Finding meaning with ‘users’, not just ‘transferring knowledge’

I personally dislike the word ‘users’, as it conjures up images of some transactional process whereby academics apply their research to a waiting horde and *TA-DA* the public cry tears of joy that their lives are better. I’m not sure quite what language is better (very happy to be advised!) but whatever we use, we need to shift away from simplistic views of impact-as-productisation and impact-as-consumption. Impact can and must always be measured at the level of the beneficiary/ies, in terms of what is meaningful for them.  As we move forward, we’re seeing co-production and patient and public involvement weaved more substantially into research delivery, and this shift of focus away from blunt assessment currency can only be a good thing.

Disrupting simplistic, linear models of ‘ultimate’ impact

Logic models of impact are great (and endorsed fabulously elsewhere) but they can unintentionally reinforce an expectation of linearity. Input-output-outcome models routinely position impact as the ultimate endpoint of a research journey. Fine. Well not fine, actually, if that shifts assessment and thinking to so far down the line that it blinds us to those meaningful effects along the way.  I’ve written before about the dangers of focusing only on ethereal and magical endpoints but if we can truly understand the iterative and co-produced nature of impact, we can make some very meaningful changes indeed.

Valuing and investing in impact professional development or ‘There are people that do this stuff? Like, willingly?’

I’m extremely proud to lead the ARMA Impact Special Interest Group, a group of over 750 research managers with a core, or connected focus on impact. This community of practice grows every year in both size and combined expertise, and offers the most incredible combined set of insights into what it’s like to ‘do impact’. Yet REF risks painting a simple one to one picture of academic (project) to impact, leaving brokers and research manager invisible in impact discourse. Impact needs a strong and valued community of practice, both in terms of dedicated impact professionals and the integration of impact into academic career development. If we don’t acknowledge the skills, efforts and expertise inherent to driving research into practice, quite how much impact can we really expect to have?

Impact does not, and never has equalled REF. Impact is not about assessment compliance, nor is it about demonstrating effects at scale at the expense of meaningful change of any size. We need to recognise impact may be driven by REF, elevated by REF, or conversely deprioritised if it doesn’t ‘fit the REF mould’.

Impact already exists beyond REF.

So perhaps the question shouldn’t be what impact looks like beyond REF, it’s what you’d do even if it wasn’t being measured.

*This blog accompanies a plenary at the Making a Difference: Impact in the Social Sciences’ Conference, April 19th 2018, St Anne’s College, Oxford*

Coming to INORMS 2018? Feeling impact’y? Go on you know you want to

Are you heading for INORMS in June 2018? Are you secretly thinking about impact, and all the joys it holds?

Well, the first step is to admit it. Go on. Treat yourself.

Just as with ARMA conferences in the last few years, INORMS offers a fantastic opportunity to connect with the impact community. If you’re new to impact or a battleworn veteran, you are extremely and equally welcome. More so even than that, you are cordially invited to the Impact Special Interest Group session (Tuesday 5th June, 11.30-12.45, co-hosted by the completely wonderful David Phipps). The session is an opportunity to connect with those working in the same space, discuss areas of challenge/opportunity and to find ways to deliver impact in your own settings. Do consider coming along. There’s no wine/coffee but I’m fairly sure there’ll be flipchart paper and come on who doesn’t love that?

So go on, admit the impact love and indulge yourself. You know you want to.

See you there!

PS. If you’re new to impact/ARMA/INORMS, feeling daunted or just want to connect ahead of the conference feel free to drop me a line.

Impact, funders and ‘joining the dots’

I’ve just spent a fantastic afternoon in the company of representatives from a range of medical research charities and NIHR. The AMRC Impact coffee club – an informal name for a truly valuable group – provides a platform for AMRC and NIHR colleagues to share practice, discuss impact-related challenges, and move as a community towards ‘better impact’.

As you can imagine I was extremely pleased to speak about the challenges for academics (including unsurprisingly REF pressures, competing impact drivers and broader issues around professional development). But this blog isn’t about that….

The main thing that struck me, which strikes me every time I meet with funders, is the strength of commitment to research and effect. Now that sounds obvious of course, but it’s so easy for academics to default to seeing funders as faceless architects of those up-til-4am-with-copious-amounts-of-coffee-to-finish funding bids.

The thing is, everyone I meet from the funding community is driven by what *their* research can achieve. For science. For society. For researchers. For patients…..there is endless behind the scenes activity to amplify both new knowledge and new practice. And it’s not surprising that the medical charities present today have that weaved into their DNA.

I was also reminded however of the challenges of drawing together funders, academics, research managers, patients, carers, policy makers (*complete your own list here*) when the simple ‘impact’ word looks and feels so different for each. I talked today about how even when we’re committed to real world change, as academics we often face the stark choice between chasing impact or the high-quality-paper-which-will-get-us-promoted. From a distance these things don’t look in conflict, but in that microsecond between grant writing / marking essays / picking up workload in an understaffed department, that choice is very real. Ultimately however much you value social change, you’re going to prioritise paying the mortgage.

So where do we go from here? There is a HUGE appetite from the NIHR and AMRC to establish/ reinforce good working practices which support academics not add to the burden. For this to happen we need to establish better dialogue, translate our linguistic shorthands (User? Patient? Stakeholder?) and establish fair impact currencies.

Let’s join the dots. I think something rather amazing might happen

Knowledge broker competencies across the institution

As I sit here at the ARMA Advanced Routes to Impact training, I’m reminded that whilst having done a blog about impact literacy, I haven’t done one on knowledge broker competencies. So I’ll pull my finger out now…..

I’m sat listening to two fascinating talks by Anthony Atkin (University of Reading) and Jacqueline Young (Lancaster University) about institutional delivery of impact and considerations for skills localisation. ie. isolated impact officers doing everything vs. institutions harnessing skills across the university. The talks make mention of the impact literacy paper, but more so the knowledge broker competencies paper. In summary, this is a framework of 80 competencies in 11 categories which reflect the breadth of skills needed to deliver impact. Here’s a swanky looking diagram (thanks Research Media)

KMb competencies.png

There is a HUGE risk that if the diversity of skills aren’t recognised, (a) many impact officers will be working in hugely isolated roles, (b) competencies aren’t appropriately balanced across the organisation, (c) there is no investment in development and (d) impact is neither sustainable nor upscalable.

If we are truly committing to impact beyond short term assessment windows we need to review how it’s delivered (that’s not a dig at REF, it’s quite clear from Catriona Firth‘s talk that actually whilst the REF process is mechanistic, there is a far deeper commitment to longer term benefit through research).

So, impact colleagues, institutions and sector leads……take a step back and look at really what’s needed to deliver impact. Cut through the points of isolation and extend the impact love (ok skills) across the organisation. Surely that can only be a good thing?

Lincoln bound!

2018, and time for a new chapter……

I’m completely delighted to have just been appointed to a new and exciting position at the University of Lincoln. From May 2018 I’ll be the new Director of Research Impact Development, a post which will work across the university to build impact strategy and strengthen impact literacy.  Not only will I be able to support impact in its broadest terms, but I’ll maintain a research component (to keep me rooted in my academic discipline) and continue to connect into the wider research sector.  Basically I’ll be me, being me, for Lincoln, who seem to find that a useful combination.

It’s genuinely fantastic to see an institution taking such a forward looking view of impact.  This post embraces the value of impact beyond REF, and focuses on developing a positive impact culture.  It is to Lincoln’s credit that they have created an innovative post which supports research informed practice, recognises the value of blended professionals, and purposefully commits to driving impact with all its associated complexities.

It is in an enormous privilege to be welcomed into a new institution with such openness, and I’m excited to work with my many new colleagues. There’s already a huge wealth of expertise across impact and broader research management at UoL and I hope I can add something useful.

And before you ask, yes Lincoln are aware of the over excitable handwavy-ness, and they still want me. Turns out Lincoln are hardy people.

Impact and Patient Involvement – what’s fair?

We all know public and patient involvement (PPI) is fundamentally important to health research, but what does it mean for impact? How does it challenge our simple definitions of impact? And how can we most meaningfully articulate the benefits of patient involvement in driving real world benefits?

Yesterday I attended a fantastic first meeting of the NIHR Working Group on Public Involvement and Engagement Impact. This group, convened and led by Simon Denegri (@SDenegri), is tasked with unpicking these and further issues around the PPI-impact link. Consisting of a wonderful assembly of PPI leads, impact people, Department of Health strategists and – most importantly – patient representatives, the working group is a space to challenge, agree, disagree, champion and have those vital lightbulb moments only possible through shared learning.

The working group leadership will draw together the broad and energetic discussions into some coherent summaries(!), but ahead of that, I think it’s fair to say:

  • There is HUGE commitment to meaningful patient involvement, and excellent work going on across the country to strengthen how this is done in practice
  • There is an undoubted challenge in knitting this together with the ‘impact agenda’ within the research sector, and a need to avoid simplifying PPI as just a pathway towards something measurable. PPI does something far more, has effects which are far less ‘tangible’, and cannot be decoupled from the value to patient, researcher and research alike.
  • Ultimately there is a moral and ethical right for patients (and their carers) to have their voice heard in research which affects them. We cannot and should not dilute their voice and instead need to look to our research processes to see where we can improve this relationship.

Where that gets us for impact remains to be seen. But I’m definitely looking forward to working it out.

Chasing the impact unicorn

Impact is big. Really big. You just won’t believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it’s a long way down the road to the chemist, but that’s just peanuts to impact.*

Nah.

Impact is change. Simples. We just want it to be mind-bogglingly and intergalactically big because REF and funders and careers and stuff.  But sometimes we need to just go back to basics and think about what change really means.  And what it means for those our work reaches.

Perhaps we need to stop chasing some fantastical possibility.

Perhaps we need to stop chasing some mythical impact unicorn.

If only someone could write a blog (for NIHR) on that oh no wait I have how handy go look here right now.

Enjoy!

* Kindly borrowed from Douglas Adams. I hope he’d approve

REF impact consultation 18/1/18: spoiler, no answers yet.

Yesterday I took part in a REF Impact consultation event in London. This workshop formed part of the broader consultation REF are undertaking with the sector to iron out some of the remaining issues following the ‘final decisions’ announced in November.  In attendance were several of the main panel chairs, sub-panel chairs, senior university staff and impact partners (eg. assessors from 2014)

The two key foci of this particular event were additionality (ie. how to accommodate continuing case studies) and expanding the underpinning research base (ie. addressing concerns over limiting case studies to a linear connection between ‘project’ and ‘effect’.  Let me start by saying immediately we don’t yet have the decisions about these.  The discussions reflected the complexity and implications surrounding these issues and doubtless REF have a huge job on their hands to wrestle with the breadth of feedback and areas of dissent.  This won’t be easy and it will have to balance a myriad of considerations.

However, several core messages came through strongly:

  • The REF team and main panel chairs were fully in agreement about their commitment to make the assessments fair and rigorous, and with recognition of the challenges/game playing last time.
  • There was general consensus that irrespective of whether something is continuing or not, the key question should be does it stand up as a case study in its own right?  Fairly ‘spirited’ discussions happened about how to articulate continuing impact, whether it should even be flagged, how it would be assessed etc. Ultimately everyone seemed to settle on simplicity and fairness being the main principles and that a continuing case study (or whatever it should be called) should be measured on its own merits.
  • The implications of broadening ‘underpinning research’ to loosen the linear connection between research and impacts and reflect a broader body of accumulate expertise were source of deep discussion.  There was real debate around where the lines should be drawn between broader research activities (not just for instance 2* papers) and those which are more engagement in character. Whilst superficially ‘broadening’ conceptually better values the academic lifecourse, it raises significant issues for assessment, judgement, and eligibility of materials for submissions.
  • All discussions reflected the broader sector and institutional challenges around impact management, assessment, narrative construction, implications for (eg) progression, incentivising short term vs long term impact and many other issues.
  • There is clear recognition that however ‘neat’ the decisions, these sit within a complex ecosystem and must be accompanied with clear guidance and underpinning principles. With institutional stakes high, and submissions so nuanced, text-only communication cannot ‘carry the burden’ of conveying such weighty expectations and must be complemented with broader communication and outreach.

The REF team will be holding several more consultations, and will be synthesising feedback into guidance following this. I don’t envy them at all, but after yesterday I’m convinced there is a real commitment to recognising – if not being able to fully accommodate – voices from the sector.

 

 

The psychology of being a patient with a chronic vascular condition. Or ‘the joy of DVTs’

Many of you will know about my health fun and games. And by fun and games I mean multiple deep vein thromboses (DVTs), those cocky little blood clots that keep colonising my left leg. The saga can probably be shorthanded to ‘9 years ago my veins decided that blood flow was overrated and shut up shop’.

Flippancy aside, the chronic pain and mobility difficulties have been horrendous, not to mention the impact on my family. And I’ve lost count of how many times I’ve ended up sobbing mid conference because of the pain of just ‘sitting’. Thankfully, overlooking the ‘unexpected clot incident of 2016‘, I’m generally far more mobile and my limp takes longer to kick in. And my walking stick now only gets broken out for special occasions.

Anyway fast forward to June 2017. I’ve been given the name of a fantastic specialist in London whose expertise may actually help fix me, when it seemed unfixable. (Spoiler, we still don’t know if it is, but let’s hope Thursday’s MRI scan shows something positive). I go for a consultation (which led to the MRI) and the discussion takes an unexpected turn. My consultant is concerned about how psychologically and emotionally difficult vascular problems can be for patients and and invites me (as a Health Psychologist) to help. Somehow.

The first step has been writing an invited commentary for ‘Venous News’, the professional magazine for vascular practitioners. This has been a chance to headline, as a patient and psychologist, the fallout of living with a long term condition. It’s a personal account, but far from unique and certainly mirrored across other conditions. I’m hopeful it helps build understanding of the invisible side of long term ill-health.

The opportunity to be able to use this pretty horrid experience, in a professional capacity, and maybe (*fingers crossed*) have some impact is a luxury many people don’t have.  I hope this is the start of a much longer journey to support patients and those clinicians who clearly care enormously about their patients’ welfare.

You can read the piece online (pg 10) or view as a pdf. Thanks to Mr Stephen Black and Venous News for the invitation.

For more information about DVTs check out @ThrombosisUK

“We’re more than REF people” – notes from the ARMA conference Impact SIG

Ok, so let’s just agree that my delay in writing up notes from the ARMA Impact SIG session again is an ‘endearing tradition’…….

Thanks to all of those who attended the SIG session at the annual conference in Liverpool this year.  Our impact community consistently showcases the collective experience, expertise and generosity which is driving impact and research management forward. It is always a pleasure to be part of those forums.

For those who weren’t able to join us, after a short presentation by Lizzie and myself on sector drivers, challenges and opportunities for impact, the assembled impactors (is this a word? *if not quickly coins*) discussed a range of issues for impact management in the current UK climate. Click here to read the full ARMA SIG 17 Discussion notes, but by way of summary:

  • In line with last year’s discussion, there is still a sense of poor recognition of the skills and expertise needed for impact. This continues to be compounded by eclectic job titles, job descriptions and role scope, alongside the common burden of being ‘stretched’ across the university to hunt down ‘nuggets‘ of impact gold.  There is an ongoing need to resource a stronger infrastructure within institutions to both ensure adequate provision is in place, and allow space and time for impact staff to develop.
  •  A common theme was the wish to drive home the message that impact managers are ‘not just REF people’.  Nor are they just nagging secretaries. Listening to SIG discussions, the professional resolve to ‘make research useful’ is continually and abundantly clear. That doesn’t mean the community’s eye isn’t on REF (how could it not be?), but that we are impassioned and committed to something more fundamental than a seven yearly assessment cycle. Perhaps our branding needs work……
  • The changeable research landscape presents challenges for establishing impact goalposts and engaging academics into the process. Whilst we now know more about REF2021, the fairly consistent picture painted about small, organisationally dispersed impact provision suggests that our institutions are not agile.  Given that we can’t just sit waiting for new rules, we must revisit our institutional set up and consider how to most effectively continue to operate and shift gear in response to sector changes.
  • Information management continues to be an issue, and whilst different universities are investing in different solutions, there remain issues for understanding what constitutes impact ‘data’, the skills needed to appraise impact related information and technical interoperability.  Activities to support this are ongoing, with a particular shout out to CASRAI (*holler*) for their work on harmonising impact information standards. Watch this space for updates….
  • Improving impact literacy and impact culture in organisations is, in effect, a product of doing all of the above (and more).  Our community is well aware of the achievements so far, but also work left to do to draw impact more securely into the academic process.
  • We should also connect multiple networks and embrace impact consultants. This is probably meant conceptually, but I for one am choosing to assume this means physically *awaits hugging opportunities*

As ever, our Impact SIG community represents the experiences and expertise at the heart of impact management.  Perhaps the increased weighting of impact from 20 to 25% for REF 2021 will galvanise the sector into more swift action to address the points above. The challenge and opportunity now is to harness this collective knowledge and connect ourselves and our networks to make it happen.

And consultants, sorry for the upcoming volume of hugs.

Gold panning small
old gold miner