Impact and the REF: the grey areas

Published on May 26th, 2011

As Professor Peter Taylor-Gooby, Chair of the HEFCE REF Social Work and Social Policy Panel, puts it: “Impact and the REF are like the policies of the current government: you know they are important but it is difficult to work out the areas of uncertainty and the areas of certainty.”

These areas of uncertainty were aired in a Q&A session at the Developing and Assessing Impact for the REF event at Bournemouth University, many of which we’ve come across at Bulletin Academic through our own client experiences. Perhaps many of the issues will be addressed in July when HEFCE releases its guidance on REF submissions, panel criteria and working methods.

A three-month consultation period will follow and the final guidelines will be published in January 2012, with impact – measured by ‘reach’ and ‘significance’ – counting for 20 per cent of the overall mark.

So, some of those grey areas then:

-          Quality of academic research underpinning the impact

Any research that underpins impact case studies submitted to the REF has to meet a quality threshold otherwise it will not be classified. Many believe this threshold is in need of clearer definition. It is also unclear whether higher rated research (e.g. 4*) with less impact will still be a better case study than a 2* research case study which has had a much greater impact.

-          The trouble with ‘travelling impact’

One audience member Professor Samuele Marcora, Director of Research at the University of Kent’s Centre for Sports Studies, raised the issue of how a department made up of many researchers who had recently moved from other institutions could demonstrate impact effectively. His particular research centre comprises academics who have moved from other academic institutions – and who therefore cannot bring their previous ‘impacts’ with them.

Another audience member spoke of a “deep inconsistency” in impact assessment saying there was “no logic” in allowing research outputs to move with an academic while impacts cannot.

-          Concerns over co-authored research

One audience member sought clarification on the issue of co-authorship of research when it comes to assessing impact. Who ‘owns’ the research for a REF impact case study? This question becomes even more fraught if there has been shared financial outlay in achieving impact. And will co-authored papers that stretch across different departments within one institution be allowed to be submitted for REF impact assessment?

-          Issues of confidentiality

What happens if research that has demonstrated considerable impact is in fact classified, one academic asked, citing potential collaboration with the Ministry of Defence.

-          Measurement of policy impact

Is there some way of grading the impact of research on the policy process? Do references to academic research in one particular type of policy paper carry more weight than references in another?

Tags: bournemouth, hefe, impact, ref, research