Doing Data Differently: Lessons from the Results Data Initiative

December 2, 2016

     By Duncan Green     

Guest post from Dustin Homer, Director of Engagement and Partnerships at Development Gatewaydustin-homer

Development folks see magical possibilities for data-driven decision-making. We want data and evidence to improve our work—to help us reach marginalized people, allocate budgets effectively, and see which activities work the best. And it’s not all buzz; we’re getting serious about investing real resources into this development data revolution.

So here’s the question: if we’re serious about promoting data-drive decisions, what should we actually invest in? Or more specifically, how do you help someone else use data more meaningfully [or can you]?

Local governments and organizations serve people directly, so we think it’s critical to focus on the ‘data revolution’ rdi-coverfrom their perspective first.  With support from the Bill & Melinda Gates Foundation, we asked some 450 people in Ghana, Sri Lanka, and Tanzania to tell us how they use health- and agriculture-focused results data and what they’d like to do differently. We’ve distilled a few lessons that are important for the wider data-for-development community:

Getting the right data: When we talk about local-level data, we aren’t talking about much. Of course our community knows this well—we talk about needing more data, higher-quality data, and better-disaggregated data. This is all true. But the bigger point is that the data that local actors spend time on isn’t the data they need to inform real work. Collecting and reporting activity or output data for someone higher-up to (possibly) use doesn’t help a local official make a better plan or management decision. So agencies and governments need to invest more time and money in the right data—and probably request less activity/output reporting. Many local-level respondents were hungry for data on the outcomes of their work; they wanted to know what they were accomplishing so they could act accordingly. This means more investment in censuses, disaggregated household surveys, and especially civil registration/vital statistics. But at the same time, people need unique local data to answer local questions that help them deal with local issues. So instead of just driving data collection from the top, we should make more funds available for locally-driven, smaller-scale data collection.

Making data analysis worthwhile: Nearly every institution in the world struggles to use data well. Local governments/implementers are no different. The main issue is not that people don’t have data skills (even though they often don’t)—it’s that they don’t have incentives to worry about data. Imagine you’re a local government health

Digital v Analogue

Digital v Analogue

official. You don’t have many resources anyway, so a “data-driven decision” may already be something of a moot point. And even if you do have some resources to move around, and you base your decision on good evidence, no one really pays much attention. It probably doesn’t mean recognition or promotion, even if things go well. Then say you’re told that you need to base your budget on data and evidence, so you spend extra hours making a solid proposal for the coming fiscal year. In return, you get the same budget you got last year, minus 10%, and no feedback on your data-driven proposal. After a few years of that, it’s no surprise that many people told us the general consensus when it comes to data is “why bother?” (to offer another anecdote, in Indonesia, local government officials used to refer to budget allocations with a term that literally translates to “bird droppings”).

While incentives issues are hard to tackle, we need to try. Data funders should make practical investments in incentives. For example, awards, prizes, or publicity for dynamic data-users. Or getting a line about “how did this person use evidence to inform their work?” into the form used to conduct annual performance reviews for public servants. Or working with a permanent secretary to recognize and publicly praise a few high-performing data users in her agency.

Creating space for action: We did come across a few impressive people who are intrinsically motivated to use data to make change happen. For example, one dynamic district health official had observed a concerning trend: maternal death rates were high in some communities in her district, but not others. She found that the problem was resource allocation—in particular, that some communities were not supporting enough community health workers. So she presented basic evidence to the district council, comparing local mortality rates to district targets, visualizing which communities had the highest mortality rates, and tabulating the factors that contributed to these deaths.  The dilbert-on-fake-numbersresponse to this public discussion of data was loud and fast. Representatives from under-performing communities scrambled to fund more health workers. Community groups were given a mandate to educate citizens on maternal mortality issues. And over the course of a couple of years, district maternal mortality rates were cut in half.

What makes this story unique is not that special data was available, or that sophisticated analysis was done. Instead, one leader saw an opportunity to make change with data, and had the desire to do something about it. In addition to promoting incentives, our investments should create space for similar local leaders to do something different as a result of data-driven insights.

At a high level, this means that money and data need to be tied more closely together. So big donor investments in results-based financing and public finance reform should be seen as a critical part of the data revolution. But at a local level, we can find easier ways to link financial data with population and results data (like the example above). We can make small implementation grants available to local-level actors who have new ideas, based on evidence, that they want to carry out. And we can offer training that is as much (or more) about communication and political navigation as it is about data cleaning and analysis.

So what: What this boils down to is that local development actors want to use data to make their work more effective. They need more data, but it needs to be the right data—so we should be judicious and decision-focused when funding new data collection. They need training and tools, too, but these are just stepping-stones. To make change happen, international data revolutionaries need to make bold investments that foster incentives and create space for local actors to use data to do something new.