I’m at one of those moments where all conversations seem to link to each other, I see complex systems everywhere, and I’m wondering whether I’m
starting to lose my marbles. Happily, lots of other people seem to be suffering from the same condition, and a bunch of us met up earlier this week with Matt Andrews, who was in the UK to promote his fab new book Limits to Institutional Reform in Development (I rave reviewed it here). The conversation was held under Chatham House rules, so no names, no institutions etc.
Whether you work on complex systems or governance reform or fragile states, the emerging common ground seems to be around what not to do and to a lesser extent, the ‘so whats’. What can outsiders do to contribute to change in complex, unpredictable situations where, whether due to domestic opposition or sheer irrelevance to actual context, imported blueprints and ‘best practice guidelines’ are unlikely to get anywhere?
In his book Matt boils down his considerable experience at the World Bank and Harvard into a proposal for ‘PDIA’ – Problem Driven iterative adaptation, which I described pretty fully in my review. The conversation this week fleshed out that approach and added some interesting new angles.
PDIA needs funding, but not big million dollar cheques that come with all the paraphernalia of targets, milestones, logframes etc that are more likely to kill thought than promote experimentation and learning. Instead, it needs a trust fund approach – lots of small grants that allow incubation of local solutions to a given problem while ‘avoiding a premature results agenda’.
But does that mean that institutional reform should avoid the big aid dollars altogether? Matt thought not – he portrayed PDIA as a new and extended incubation phase, which can then take the homegrown solutions that emerge and move into the more traditional aid world of large scale, large budget programming. So the challenge for aid agencies is how to create, fund and protect a space within their institutions for small budget experimentation and incubation, sitting in parallel with the big stuff.
Timelines emerged as a useful, but undervalued tool. But these are timelines of what has actually happened in the past, not the imaginary future timelines of funding applications. Matt reckons any project seeking funding should start by building a 20 year timeline of what has happened on that issue/in that locality. If done properly, the exercise of reconstructing the timeline using documents and interviews will reveal overlapping interpretations of what actually happened and recover the kinds of knowledge and experiences that all too often go missing in Aid World as staff leave and projects are wound up. We need a decent timeline methodology – Matt uses the work of Peter Hall at Harvard but it also sounds a lot like process tracing, something our MEL team uses.
The issue of narratives is central – it lies at the heart of the response to a reductionist results agenda that privileges pseudo medical trial data over real experience. Claire Melamed likes to say ‘the plural of anecdote is not data’. True, but I think that a well researched anecdote rapidly becomes a ‘narrative’, and the plural of narrative can definitely be evidence, if not data. Matt, ODI and Oxfam are all separately thinking about the need to build a collection of rigorous, nuanced narratives on stories of power and change – we’ll be swapping notes and hopefully coming up with some ideas for working together on this. What would people recommend in terms of references on rigorous narrative methodologies?
There was a good discussion on what constitutes ‘results’. Good PDIA-type work in developing countries requires a rapid feedback loop of results, but of a different kind to those typically demanded by the aid business. Developing country politicians want to know what’s happening with their money, what has been learned, what has worked and what hasn’t, and how the project has responded. They don’t need the (often bogus) certainty and data demanded by aid planners.
I do find this all slightly baffling – politicians intrinsically know how to navigate in complex environments, respond to shocks and opportunities, using trial and error, instinct and rules of thumb. They make decision on partial information and change direction if things don’t work. That’s what politics is about. But then they become aid ministers in donor countries, and suddenly buy into a paraphernalia of logframes and a particular understanding of
results that in some other part of their brains they must know has huge limitations in the real world. How to get ministers to think more like pols and less like aid bureaucrats?
All fascinating and thanks to Matt for kicking off and CGD Europe for organizing the discussion (am I allowed to say that under Chatham House rules? If not, please ignore). I’m thinking of writing a paper on the ‘so whats’ of complex systems, but will first wade through the draft of Ben Ramalingam’s forthcoming book before deciding whether it’s necessary.
Update: more thoughts from Matt Andrews on his blog