Guest post from David Rinnert (@DRinnert) and Liz Brower (@liz_brower1), both of DFID
Over the last decade there has been significant investment in high-quality, policy-relevant research and evidence focussed on poverty reduction. For example, the American Economic Association’s registry for randomised controlled trials currently lists 1,294 studies in 106 countries, many of which have yielded insights directly relevant to the SDGs; there is an even larger number of qualitative evaluations and research on development interventions out there. However, as shown again and again (including on this blog), the generation of relevant evidence alone does not lead to improved development outcomes. In many countries and sectors, lack of demand for and use of evidence by decision-makers remains an issue. Policymakers, especially at lower levels, are not rewarded for innovation, and they lack the capacity and/or incentives to use evidence for decisions.
The UK Department for International Development (DFID) designed the Building Capacity to Use Research Evidence (BCURE) programme in 2013 to address this issue across 12 countries. Over the past four years, BCURE has promoted the use of evidence by decision makers, which has contributed to improved development outcomes and has generated many lessons on what works and what doesn’t in the promotion of evidence uptake. However, a key question for BCURE partners has been how to think more precisely about the increased use of evidence as an improved development measure, and how policymakers, researchers and practitioners should value the impact generated and communicate this in an effective way. There is a substantial body of literature discussing the value of research and evidence products (for example see here, here and here, or here on FP2P); however there has been less focus on unpacking and valuing evidence use. Building on existing literature and experience from the programme, BCURE has developed a typology for the type of evidence use by policymakers, namely: Transparent Use, Embedded Use and Instrumental Use, as defined in the table below. The framework can be used to capture the range of uses of evidence and offers a starting point for recording and comparing value through the avenues: scope, depth and sustainability.
Figure 1: BCURE Value of Evidence Use Framework
| Transparent Use | Embedded Use | Instrumental Use | |
| Description | Increased understanding and transparent use of (bodies of) evidence by policymakers. | No direct action is taken as a result of the evidence, but use of evidence becomes embedded in processes, systems and working culture. | Knowledge from robust evidence is used directly to inform policy or programme. |
| Examples
|
BCURE VakaYiko: Several roundtables were held to help bridge the gap between research and policy making on climate change in Kenya and to help decision-makers acknowledge the full body of evidence on climate change in the country. | BCURE Harvard: the researchers worked directly with government technicians to create a Report Dashboard designed to serve as a one-stop shop for over 50 indicators deemed crucial for evaluating MGNREGA. | BCURE University of Johannesburg: In South Africa the evidence map, published by DPME, fed directly into the decision-making of the White Paper on Human Settlements. |
| Scope: The array of policymakers impacted by the reform – is its impact far reaching across actors? | +++ inter government, policy teams and country offices | + One local government ministry | +++ national level policy |
| Depth: Impact of change, how large is the size of the reform (for instrumental use it could eg be population reached)? Is there a substantial change from previous practice? | + No in-depth change in practice that would be directly attributable to BCURE but contribution to a set of follow up actions | ++ Evidence tool created and saw immediate use, 150,000 hits in the first year. | ++ (tbc) The Human Settlements Policy is potentially reaching a large proportion of the population, however, overall effect is yet to be determined based on M&E results |
| Sustainability: How sustainable is the change in the use of evidence? | + One-off meetings but with potential to influence further changes in the use of evidence | ++Evidence suggests this will be a prolonged change | ++ Evidence used for several policy decisions with potential to influence further policy choices |
One example of value created from evidence use comes from India. A BCURE implementing organisation – Evidence for Policy Design at Harvard Kennedy School – saw that the Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA) was producing vast amounts of data that could be used to monitor programme performance and spur improvements, but that, in reality, ended up sitting in an unused website database. With BCURE funding, the researchers worked directly with government technicians to create a Report Dashboard designed to serve as a one-stop shop for over 50 indicators deemed crucial for evaluating MGNREGA, an example of value generated through Embedded Use as illustrated above (full case study here). This tool saw wide instrumental use as well: over 150,000 hits by over 100,000 users in the year following the launch.
In South Africa, the University of Johannesburg through BCURE worked with the Department for Planning, Monitoring and Evaluation (DPME) to develop a policy-relevant evidence mapping tool. The evidence map offered the department visual and easy access to evidence. Other policy stakeholders benefited from the lessons learnt from the pilot evidence map – an example of Embedded Use discussed in the table. A range of public sector actors were invited to engage with the methodology (Transparent Use). The evidence map, hosted by DPME, fed directly into the decision-making of a White Paper on Human Settlements, an example of Instrumental Use of evidence.
While many BCURE projects achieved positive development outcomes, even the less successful pilots have helped unpack some of the main challenges and barriers to evidence use. One of the key lessons is that we need to better understand politics and incentive structures in government organisations when promoting evidence uptake. Even if individual or organisational capacity exists, evidence may remain unused if groups with sufficient bargaining power lack incentives to improve policy outcomes. Promoting the use of evidence thus requires an in-depth understanding of the political economy, and a politically savvy implementation approach. Furthermore, BCURE was more successful where it helped coordinate between the often large amount of existing research organisations and projects, rather than creating new structures or mechanisms.
The examples and lessons from BCURE illustrate the variety of ways in which we can support the use of evidence by policymakers, and how this in turn can contribute to improved development outcomes. Our framework broadens the focus of evidence use above and beyond ‘instrumental use’, and lessons from this work highlight the importance of thinking and working politically in this area.
We would love to hear from other organisations on how they are tackling the evidence use puzzle. What else could and should be done to get more research and evidence into action in development?
David Rinnert is an Evaluation Adviser and Evidence Broker with the UK Department for International Development (DFID). Liz Brower is an Assistant Economist with DFID. The views expressed in this blog do not necessarily reflect the UK government’s official policies.
All reports, learning documents and other resources from BCURE can be found here.