This fascinating excerpt from a recent Owen Barder speech to the little-known-but-huge Children’s Investment Fund Foundation (CIFF) covers two new papers 
on the management of development interventions, with big potential implications:
‘[First] a study of the evaluations of 10,000 aid projects over the last ten years from nine different development organizations. In this paper Dan Honig, from Harvard University, looks at whether different kinds of projects have been successful in different contexts, and he looks at the impact of organizational devolution within aid agencies. He takes all this data and does some regression analysis to try to determine the factors that affect the success of aid projects.
Dan finds, as you would expect, that aid projects are much less likely to succeed in complex or fragile environments, such as in post-conflict countries, and that more complex projects are less likely to be successful than simple projects.
So far, you don’t really need a Harvard academic to tell you this. The interesting part of his findings is that aid agencies that allow a large degree of autonomy to their people on the ground and in implementing agencies see a much lower decline in performance for projects in a complex environment than do agencies which exert a higher degree of monitoring and control.
According to Dan’s numbers, USAID projects scores are about 20% worse in fragile countries than in more simple environments. But if USAID had the same amount of organisational autonomy as DFID, Dan’s results suggest their project success would fall by only about 2% when they work in fragile countries. So Dan Honig’s paper confirms what you might expect: more likely it is that things will change in unexpected ways, the more important it is to have power and decision-making sit with the people who can see that change coming and respond to it.
The second paper I want to tell you about was written by Imran Rasul and Dan Rogger from University College London. They have assembled an extraordinary dataset of 4,700 public sector projects in Nigeria. They have hand coded independent assessments of the projects’ completion rate and quality, and the complexity of the project; and they have conducted a rigorous survey to quantify the management practices of the 63 different organisations responsible for those projects.
Rasul and Rogger also find that more complex projects have lower success rates than simple ones, which is what you would expect. And they also find a strong statistically significant effect from schemes to create incentives for the bureaucrats and to measure performance – but these effects are negative, not positive, and much more negative for complex projects than for simple ones. Rasul and Rogger have some quite detailed information about organizational incentives, and so they are able to provide some evidence about what matters.
In summary, they find that freedom to adapt and respond improves results for most programmes but most especially for complex projects. They find that incentives schemes make very little difference either way – presumably because the organisations carrying out the work are motivated by intrinsic motivations. And they find that performance monitoring has a significant negative effect on results. Rasul and Rogger show that there is a modest complementarity between incentives and autonomy. They get a correlation coefficient of about 24%. One interpretation of this is that where organisations are able to measure results, they are more likely to be willing or able to grant autonomy to the implementing agents.
The important thing in their data, however, is that it is the autonomy, not the results measurement, which is bringing about the improvement. It follows that the results agenda is likely to improve project effectiveness when it is used instead of micromanagement of inputs and processes, but likely to make little or no difference if it is used on top of that micromanagement, as has often been the case with official aid agencies in recent years.’
Sounds plausible – does that resonate with a lot of aid agency staff outside HQ? Over to you