Where’s the evidence? A systematic approach
In June, Defra published for the first time a comprehensive evidence strategy covering the whole of the department and its network of agencies and Non-Departmental Public Bodies. In his introduction, Professor Ian Boyd, Defra’s Chief Scientific Advisor, stated that the strategy will “require changes in the way we commission, collect and use evidence with greater participation from, and collaboration with, external partners and providers of evidence”. Included within the strategy is a recurrent emphasis on innovation, including finding ways to make better, more efficient use of the data and evidence that already exists.
Last week saw a room full of policymakers, academics and knowledge exchange experts gather for a NERC workshop highlighting one way in which Defra’s evidence strategy is being put into practice. The workshop introduced the use of evidence reviews as a method for facilitating evidence-based policy, concentrating on the work of the Join Water Evidence Group (JWEG), which brings together teams from across the Defra network to make better use of the available evidence in land and water management.
What is an Evidence Review?
“Evidence review” is an umbrella term for a range of methods for collating and synthesising evidence. In its most basic form, this may comprise a simple literature review, traditionally the most common approach to providing an overview of a topic. However, an increasing recognition of the potential for bias and subjectivity, and a lack of transparency, in conventional literature reviews has led to a growing interest in more systematic approaches that can better inform decision making and meet the evidence needs of policymakers.
Drawing on an approach developed to asses medical interventions by the Cochrane Collaboration, a full systematic review aims to conduct a thorough search of both published and grey literature according to a pre-defined, transparent and replicable search protocol and screening process. This map of research is then subject to a critical synthesis and analysis. In applied ecology, the Collaboration for Environmental Evidence has led the way in adapting this methodology for environmental research, and publishing systematic reviews on a host of topics.
While a full systematic review meets the “gold standard” for rigour, objectivity and transparency, it is also time consuming and resource intensive. To that end, the UK Civil Service has developed two related methods that follow systematic principles, but can be carried out more quickly and less expensively: the quick scoping review, and the rapid evidence assessment.
Quick scoping reviews are best employed to investigate emerging or low risk topics, retaining the use of a pre-defined search protocol but with a limited scope and focusing on describing rather than critically analysing the existing evidence. A rapid evidence assessment plots a middle path, retaining the rigour and transparency of a systematic review, including the use of grey literature and an appraisal of the relevance and robustness of the evidence found, but limiting the breadth of the search and depth of critical analysis.
Evidence Reviews for Policy Making
As Defra, and other Government departments, seek to maximise the efficient use of the evidence from existing research in order to inform policy and decision-making, it appears that systematic reviews will play an increasingly important role. At the workshop, Dr Alexandra Collins, NERC Knowledge Exchange Fellow, outlined a number of reviews already commissioned by the Joint Water Evidence Group, on topics as diverse as the impact of the “Yellowfish” project in engaging local communities to improve the water environment, and a comprehensive systematic review of the effectiveness of on-farm measures to improve water quality. While JWEG is leading the way on the use of evidence reviews, this approach is increasingly popular across the Defra network.
With Defra commissioning evidence reviews through a competitive tendering process, there are burgeoning opportunities for institutions and individuals equipped to carry out systematic reviews to increase the impact of their research and play a crucial role at the science-policy interface.
Like what we stand for?
Support our mission and help develop the next generation of ecologists by donating to the British Ecological Society.