The questions below are a summary of queries raised in the DHP session, with some responses from me 🙂
Is theory building impact?
Impact is the provable benefit of research in the real world. Ie, the effects felt by people, business, the economy, the environment (etc) which arise somehow from our research. The way we get there is varied, connected and can be immediate or take a long time. Applied research tends to be a more direct pathway, for example with interventions being trialled or used by people, seeing benefit pretty much straight away. For research as the more exploratory or basic end of the continuum, the path is invariably more indirect. This kind of research can be analogised as providing the ‘building blocks’ of knowledge for applied research, or providing the first baton pass in the impact marathon. So is theory building impact? Not in the formal definition of impact, no. But is it a vital part of the puzzle? Absolutely yes.
What resources are available for supporting impact planning (and what does a good plan look like)?
There are so many resources now available for impact, a result of how the agenda has cemented and matured across the sector. I’ve put a range of resources on my blog post, but as a quick crib sheet:
- Impact Literacy workbook – a planning tool developed by myself, David Phipps with Emerald Publishing to work through the why, who, how and what of impact
- NIHR impact toolkit database – a searchable list of loads of impact tools. NB I can’t recommend a specific tool for a specific project as there’s so much crossover.
- Key (but not only!) journals for impact related literature are Evidence and Policy, Research Evaluation and Implementation Science.
A good impact plan is strategic (has a sense of goals and the methods to get there), is rooted in the needs of users (the ‘so what’ aspect), and strikes a balance between an achievable plan without being unreasonably ‘certain’ of what’s possible in a changing environment.
What’s the role of participatory research in impact?
Participatory research is so incredibly valuable for impact. It helps identify the base ‘problem’, shape the research process, identify any necessary ‘course corrections’ throughout the process, and ensure a meaningful line of sight to effects and ways to measure them. Not all research is participatory, so there should be no presumption of precisely what relationship is needed between academics and non-academics, but if your work needs to be ‘used’, it needs people at the heart of it. If you’re starting out, find academics who’re published in the area and follow their work / social media / training events, and look outwards other countries who’ve centralised knowledge mobilisation and co-production (eg. The Co-produced Pathway to Impact Describes Knowledge Mobilization Processes) or broader (non research) good practice for engaging outside of academia (eg. plan, monitor and evaluate participatory methods)
Whose impact counts?
A: I’ve slightly paraphrased this question as in its original form related to tensions between stakeholders and academics in determining what the focus of an intervention should be. There isn’t a single simple answer to this as there’s no single simple way to say whose voice counts most. In any situation there may be a myriad of goals people want to focus on, or think are important, and it’s like to be a process of negotiation and discussion, particularly when you don’t hold all the cards. I’d say always to centre the needs of the main beneficiary (eg patient), and fairly and accurately determine what the intervention could reasonably achieve. It’s all well and good people wanting an intervention to change the world, but if in reality it can only raise awareness/ help build self efficacy, any impact goals outside of that may well need to be achieved by other means.
How might someone scale up a case study intervention? Should you revisit the ‘problem’, and ascertain if the problem is the same in other settings first?
Simply put, if you’re ‘relocating’ an intervention to another location (eg. another service, community, venue etc), you should sensecheck if the problem and conditions are still a match. This can be light touch, for example speaking with the service manager of the new location, or a heavier duty needs assessment as suits. Checking the ‘problem is still the problem’ means you can repeat the intervention with confidence it’s addressing the right thing. Similarly by checking that the context is as conducive you can avoid unanticipated problems (eg. if your intervention requires gym access and you’ve run version 1 in the middle of a busy city, trying to repeat this as version 2 in a geographically spread rural location may not be as successful)
What top tips would you give for building impact for the next REF, and how do we best engage others who might not be aware of or interested in REF?
For so many of us, REF has been rough, and has left a legacy of a community conflating impact with assessment and hating it as a result. For the next REF we need to do a few things. Firstly, we need to heal from this one (my thoughts here!). Secondly, we need to set in place more supportive, literate and healthy institutional practices to build an inclusive environment. Thirdly, we need to recognise, and help everyone at all levels of the sector recognise, that ‘making a difference’ needs an investment in people, skills and connections with non-academia. Building engagement with impact needs to start with ‘making a difference’, and not with the agendas the oversimplify (or complicate!) what counts.
How do we best present qualitative evidence?
Qualitative evidence is so important – it shows the depth and the meaning of the change. This is almost always strongest a) using the voice of the person who benefitted (eg. quotes), b) articulated with phrasing indicating the nature/direction of the change, and c) connected back to the ‘so what’. The more we, as a community, can convey meaningful change through qualitative data, the more normalised it will be.
A common issue with interventions (especially tech based) is low usage and high attrition, which may influence efficacy. Any tips?
Thankfully there’s already an awesome paper on this: Beyond Adoption: A New Framework for Theorizing and Evaluating Nonadoption, Abandonment, and Challenges to the Scale-Up, Spread, and Sustainability of Health and Care Technologies
How well has impact been received by academic and non-academic people? What types of challenges are you facing?
Let’s say it’s a mixed bag….! Some academics love impact but hate REF. Some hate impact and can’t see why it should be applied to their research. The most heart breaking bit is when people feel they’re being told their research has no value unless it has impact, or that their impact ‘isn’t enough’. Sure REF might have specific expectations and institutions might have to pick their ‘stars’, but that is fundamentally different to a statement of the value any specific piece of research has. Some research can’t ever reasonably be expected to deliver impact in the way it’s so often simply conceived. A minor soapbox moment – notwithstanding the amazing work of those whose work is showcased in case studies, too many people are feeling inadequate because of the myths and unchecked assumptions about impact, and that can’t be right. Non-academics have been unbelievably helpful, and the REF agenda has engineered an academic community more ‘primed’ to find better ways to connect with them. But it remains a challenge to do this without placing such a burden (providing evidence) on them to sour relationships.
Julie Bayley 