Impact Science

Why Impact Is Integral To Research Design 

Baking In Impact At The Start Enhances Research Quality 

Dr Iain Coleman, who leads the work of Impact Science in the UK, joined us to participate in a series of insightful conversations. Dr Coleman provides comprehensive impact and engagement support to universities and researchers, in the context of the REF, the KEF and the broader research impact and public engagement agenda.

This week’s blog is based on Dr Coleman’s commentary on Baking in Impact Right at the Start of the Research Process

Ideally, when should universities start their process of working on and evaluating research impact?

I can’t emphasise enough that thinking about impact and evaluation needs to be made intrinsic to the process of research itself. It has to be factored in right from the beginning so that the effort involved in achieving and evaluating impact is budgeted for and allocated the right resources. Of course, this is an ideal scenario. Even without this kind of forward planning, impact and evaluation can still happen. It’s not the end of the world. But it is more effective and efficient to think about impact and evaluation as integral to the research program.

What benefits can universities gain if they start focusing on impact from the start?

The first and the most obvious benefit of focusing on impact from the start is that it is almost always easier to gather evidence of impact along the way, rather than trying to get hold of it later. You can also ensure from the start that stakeholders and beneficiaries will be willing to provide testimonials. It’s a lot easier to have that conversation right at the start rather than going back to them later with a request.

But an even greater benefit is that factoring in impact right from the start can improve the quality of the impact, and even of the research itself.  For instance, universities can set up interim evaluation mechanisms for key milestones during the course of the programme. These evaluations can help determine if the research is on the right track, and make a course correction if it isn’t, or to see that it taken an unexpected turn that might lead you to shift focus. I have mentioned this in a previous conversation as well, that co-creation with stakeholders is a powerful way of developing research and impact together. It improves the research process and also leads to stronger impact.

How can universities start this process? What are the things they should set up at this stage to evaluate impact?

The first thing is to go through an exercise of identifying stakeholders and beneficiaries and engaging constructively with them. The list need not be exhaustive and it might even change over time but having that understanding right at the start often leads to a rich engagement with those stakeholders.

The second thing to do, and I have mentioned it in response to the earlier question as well, is to map out key milestones of the project. For each milestone, universities should also consider what kind of evidence they need to gather so that they will know what steps to take next.

Impact evidence can take various forms – real-time evaluations, retrospective follow-ups, testimonials and so on, and these may need to be done at various points of time. In a few cases, some of the stakeholders involved in research may be doing their own impact gathering and evaluations. You should consider if those can be used for your purposes as well.

The last and perhaps most important thing to consider is the amount of resources that the university is going to set aside for evaluating impact. There’s a rule of thumb that about five per cent of the project cost should be set aside for impact gathering. In fact, these days that’s often seen as too little, with ten per cent or more being the right level. Assessing the effort and allocating the right budget at the beginning can make all the difference. It might be worthwhile to bring in an evaluation expert to help with impact assessment because it can get quite complex. But it is key to allocate whatever financial resources the university deems fit for this exercise upfront so that there is no scrambling around later.

What practices should universities refrain from when it comes to assessing impact?

My first suggestion on this would be: don’t get too rigid. You can plan everything down to the last detail, but research doesn’t always follow a determined path. It wouldn’t be research otherwise. Similarly, impact too can arise from all kinds of unexpected places. So yes, you do need to plan for impact – making sure the budget and resources are available – but you need to remain open-minded about what will emerge out of it. 

The second is important for budget holders and senior university management to understand. Impact is not a cost, it is an investment. It plays an instrumental role in creating and building the university’s reputation.

Specifically in the case of the United Kingdom, a four star case study is worth hundreds of thousands to the university in the REF. That is reason enough to put in the right amount of resources into it.

If a university tries to do it cheaply, they are just not going to get the results, whether in the form of impact, research funding or even reputation. But if you try to do it on the cheap, if you overstretch your staff and underfund your impact support, you are not going to get the returns. Impact is an investment that pays off many-fold.