That was the question that the latest in the ‘Talk Science‘ series of events at the British Library tried to address on Tuesday evening. The event, a panel discussion chaired by William Cullerne Bown of Research Fortnight and sponsored by the Society of Biology, set out to consider whether the current shift towards scientists and other researchers needing to specify the ‘impact’ of their work is justified, and how impact can be measured. Debate centred around the first of these points – perhaps understandably as measuring impact was recognised by all present to be very difficult.

At the beginning of the evening the panellists (Prof. Douglas Kell, Chief Executive of the Biotechnology and Biosciences Research Council (BBSRC); Chris Hale, Deputy Director of Policy at Universities UK; Prof. Geraint Rhys, Director of Neuroscience at University College London (UCL); and Prof. Nick Tyler, Head of Civil, Environmental and Geomatic Engineering, also at UCL) were given an opportunity to outline their views on impact, before taking questions from the floor. Doug Kell was clear that research shows undisputedly that science and technology have ‘impact’, on the economy and on society. Policy planning rounds and changes of government mean that the science and engineering community are called upon to articulate these impacts on a cyclical basis but, Doug Kell emphasised, being asked about the impact of your research and being invited to justify this, is not new. He emphasised that the ‘pathways to impact‘ introduced by the Research Councils are not about story-telling and predicting the impacts of your work; instead the emphasis is on outlining how you would capitalise on any impacts from your research, were these to arise.

Chris Hale again highlighted that impact isn’t new; a theme to emerge consistently throughout the evening. He stressed that weighting in academic assessment, such as the Research Excellence Framework (REF), should be given to the quality of the science, with impact ultimately weighted lower. Panellists seemed to be supportive of the 20% of the REF assessment being dependent on impact, with ’20% of this 20%’ being an assessment of organisational culture to support a researcher aiming to build upon and generate impact for their research. All felt that the institution, rather than the individual, was the correct unit on which to base an assessment of impact.

Geraint Rhys suggested that ‘impact is hard to define, but I know it when I see it’. Here he captured the major issue at the centre of discussions around impact; how can it be measured? All of the researchers on the panel had examples from their own work of discoveries which had eventually had impact. Doug Kell mentioned that his early research had fed into the development of a spin-out company which is still trading, for example. Issues arise however when you are a researcher in a branch of science, like ecology, which doesn’t lead to spin-outs, the development of ‘widgets’ and gadgets. There was no discussion of this at the event, given limited time, and it would have been interesting to reflect further on this issue.

There was some interesting discussion around the involvement of the public in setting measures of impact and in evaluating the outputs of research. Consensus seemed to be that the public should be engaged in a two-way, meaningful, dialogue with scientists but that the public should not be ‘put in charge’ of directing research priorities. The views of the public should be taken into account, but no one interest group should be allowed to dominate discussion. The panel felt that the current emphasis on impact in research would lead to greater public engagement with science, and that the media was a vital conduit for scientists to communicate the significance of their work. Equally important, Geraint Rhys emphasised, is the culture at research institutions. Many researchers are keen to communicate the results of their work – many scientists have entered the field because they want to ‘change the world’. Yet many lack the skills, and in some cases the right partnerships, to communicate effectively. The culture at universities, encouraging and supporting researchers in their outreach activities, is very important therefore.

The point raised by the final questioner at the end of the night illustrated why a follow-up Talk Science event on impact would be extremely valuable. Stressing that he’d heard nothing about how to measure impact the questioner suggested that researchers and funders should be honest about the ‘fluffiness’ of the concept of impact. Although recognising the examples given by Doug Kell and others on the panel regarding the impacts of their own work, he suggested that these were ‘exceptions’ and that it would be very dangerous to make policy on this basis. The teaching and mentoring that he undertakes of undergraduates at his institution, he stated, are not recognised in measures of impact, despite the clear benefit that educated citizens contribute to the economy and society as a whole. A much broader definition of impact is therefore needed.