A couple of weeks ago I posted a fairly polemical piece about the hype around âpayment by resultsâ, which prompted quite a response, including a piece by CGDâs Nancy Birdsall and William Savedoff, and an excellent set of comments from a bunch of people who are much more on top of the issue than I am (not difficult, I know).
Nancy argued that the problem with PbR is that it is mostly a severely bastardized version of CGDâs original Cash on Delivery proposal:
âPbR programs tend to create âdeliverablesâ defined by donors, to be delivered on a preset schedule, which invites creation of âplansâ and âresult chainsâ and fixed implementation schedules. That, in turn, diverts donor attention from outcomes to inputs, indulging donor impatience and discouraging the kinds of local initiative and innovation (e.g., Problem-Driven Iterative Adaptation) that are ultimately the best guarantee of sustained progress.â
Which raises an interesting point that the aid biz seldom thinks about â any clever idea is likely to be mutilated by institutional practice, culture, incentives etc on the way to being implemented. Is half a good idea still good, or is it likely to be even worse than what it is replacing?
What emerged from the comments thread is that there is a lot of experimentation going on, generating a growing understanding of doâs and donâts, and a few surprises along the way. Here are some of the themes that emerged from the discussion on the blog, and within Oxfam.
Could PbR encourage downward accountability to poor people and communities rather than just upward accountability to donors?
Michael OâDonnell, BOND: âTantalisingly, at a meeting hosted by WorldVision last week, we heard of communities who were involved in verification of results from PbR projects feeling that NGOs were much more accountable to them in that PbR project than in other work. There is scope to do this in a clever way to promote agendas like adaptive management, downwards accountability â but it takes a lot of thought.â
Paul Harvey, ODI: Â âAre there any examples of payments based on customer satisfaction type ratings? So, in humanitarian aid, you could have a bonus for agencies where disaster-affected people rate the aid provided on measures such as whether it was fairly targeted, delivered with respect to dignity, consulted fully with local people, timely, adequate and appropriate.â
Does PbR encourage learning and better Monitoring and Evaluation?
Michael OâDonnell, BOND talked of âthe positives that we now hear from organisations implementing PbR. The focus on results is helping improve M&E and performance, but within the narrow scope of whatever results are defined as triggering payments. However, those benefits may not spill over into wider M&E systems and learning beyond the project.â
Oxfamâs Francesco Rigamonti, who is running a huge (ÂŁ20m) DFID PbR contract in Eastern DRC and Kenya on Water, Sanitation and Hygiene (WASH) is even more positive âPBR allowed us to strengthen our M&E and make it more rigorous and this can have positive effects on other programmes implemented with other funding mechanisms.â
But Francesco also sees dangers that PbR will push agencies into âcounting what can be countedâ rather than âcounting what countsâ. Francesco wrote a nice blog on his experience of PbR, which I had missed.
The PbR contract with DFID has an interesting mix of short term and longer term indicators: âIn the first phase we had to deliver two or more WASH services to about 850,000 people. Starting from today we have entered a new phase in which our attention will be solely focused on ensuring sustainability and sustained behavioural change for the beneficiaries reached so far. DFID will stop paying us every quarter for the number of people with access to a latrine or safe and clean water. From now on we will be paid once per year, and the actual level of payment will be based on a survey showing how many people continue to have access to water and healthy sanitation and show signs of sustained behaviour change.â
PbR is a rapidly expanding and diverse field
Donald Menzies of DFID stressed the diversity the sectors in which PbR is now operating, including low carbon energy, girlsâ education and agriculture.
Risk of crowding out smaller players
According to Oxfamâs Helen Bushell: âWe are already seeing a risk of crowding out. At a meeting in Whitehall 2 weeks ago for an upcoming ÂŁ100 million plus call (WASH 2020 Challenge programme) it was surprising how few agencies were in the room. The combination of the financial risk transfer, cash flow implications, management costs and risk management itself and the costs of learning and adapting are arguably beyond the risk appetite of many agencies. At a time when DFID is looking to promote southern civil society, and advance small and medium NGOs this funding model would seem to be working the other way.â
A Mixed Bag for NGOs?
Oxfamâs Head of Programme Funding Tom Winslow emailed: âWhile I agree with many of the criticisms that you’ve identified in your latest blog, the evidence from our own experience in Oxfam is that it does produce some positive results – greater flexibility in programme spend to achieve outputs (which programme teams really value), greater ability to cover the full costs of implementation (because we charge the donor a price rather than submit a budget), much greater incentive to spend and deliver on time (which can only benefit people in poverty), etc.â
Based on Oxfamâs experience, Emma Feeny pointed to a useful checklist for NGOs considering implementing a Water, Sanitation and Hygiene (WASH) programme on a PbR basis.
My takeaways?
I still think the hype curve is a useful construct for PbR and any other aid fad, but we seem to have all sections of the curve happening at once: the snakeoil salesmen are out there over-selling; the bah humbug types like me are pouring cold water on it; and loads of experimentation and learning is already propelling us towards a more realistic grasp of where/when PbR might be useful and how it needs to evolve (the plateau of productivity), much faster than the comparable curve for, say, microfinance.
What have I missed?
And here’s today’s vlog – lots of advice on my first one last week (thanks to all). Result: I will do these as a supplementary to original thinkpieces (not book reviews or guest posts, obvs) and don’t worry, they are not going to replace the written version. Any more feedback v welcome – done like this, they don’t take much time, but I’m still not sure how much they add to text blogging.