Book Review: Navigation by Judgment, by Dan Honig

As its subtitle, ‘Why and When Top-Down Management of Foreign Aid Doesn’t Work’, suggests, this is an addition to the growing library of books on aid reform. And a very useful one.

Honig is a hybrid scholar-practitioner, with dirt under his fingernails in East Timor and Liberia, and the book is for aid insiders, whether practitioners or scholars, focusing on ‘the internal organization of international Development Organizations’ (IDOs), It explores how that internal organization interacts with different kinds of issue (health, infrastructure, governance) or context (stable, predictable, messy & chaotic).

In doing so, Honig attempts to make his arguments convincing to both quals and quants, mixing in depth case studies (4 projects each from DFID and USAID), and number crunching on a mammoth dataset he has built of 14,000 aid projects (which he’s made open access – kudos!).

It’s that addition of a big number crunch that probably adds the most value, because it confirms the arguments and findings on Doing Development Differently, Adaptive Management etc etc, but in a way that is more likely to satisfy, or at least silence, sceptics. Experience is on our side, and now the numbers are too!

The summary of the summary is:

‘This book is an exploration of the costs and benefits of top-down control as compared to relying on the judgment of field agents. I argue that just as there can be too little control, there can also be too much. In doing so, I echo the view of no less an authority than former USAID administrator Andrew Natsios, who has argued that the IDO he used to run suffers from “Obsessive Measurement Disorder (OMD)”, an intellectual dysfunction rooted in the notion that counting everything in government programs . . . will produce better policy choices and improve management.’

To unpack this he uses my favourite device – a 2×2. On the X-axis axis he has level of project verifiability – can you measure results on something useful like whether people’s nutrition improves, or do you have to measure ‘outputs’, eg number of training seminars, which are a poor proxy for actual results? On the Y-axis we have high environmental predictability v low. It’s pretty similar to the one we came up with in a USAID/DFID seminar a couple of years ago, and helps the discussion forward by showing that in some quadrants logframes and conventional approaches work best, in others we need to ‘do development differently’.

He summarizes his core argument in the academic language of principal-agent theory:

‘Using Navigation by Judgment requires relying on agent judgment, and even the best agents will sometimes make mistakes. Navigation by Judgment is a second-best strategy—a strategy to employ when it is less bad than the distortions and constraints of top-down control.’

So what are his findings? The biggest contribution is probably the way organizational structure and culture influences aid. Honig distinguishes between ‘more politically insecure’ IDOs, constantly looking over their shoulder and pacifying their critics (USAID and Congress) and less insecure ones that have the confidence to try stuff out (DFID – though that may come as news to some DFID people!). The former spend huge amounts of energy trying to control what happens in the field, and gathering bucket loads of numbers to chuck at their enemies political bosses. The latter have more latitude to think about what kind of monitoring, evaluation and learning might actually help a given project achieve its aims.

This matters because ‘As environments become more unpredictable and interventions less externally verifiable, the flaws of principal control weigh IDO projects down more heavily than do the weaknesses of fallible agent judgment. When the going gets tough, Navigation by Judgment helps cope with the rougher terrain.’ So in messy places like fragile/conflict states, IDOs that navigate by judgement do better than their more rigid colleagues, even though both groups find life harder than in stable places.

Unfortunately he stops there – I would have liked to see some discussion of whether USAID and other more under-the-cosh agencies can do much to remedy their situation, what political moments, arguments or alliances might allow them to make progress, or at least prevent rapid backward movement. Or whether DFID is likely to stay as insulated as he currently thinks it is.

Instead he slips back into a more conventional set of policy recommendations on how to make ‘IDOs fit for purpose’:

• measure smarter and differentiate between measurement for learning (how to improve the project), and measurement for accountability (feeding the wolves)
• change the frequency of measurement (quarterly results can kill a project and leave its staff largely braindead)
• think about kinds of accountability other than voodoo numbers, eg peer judgement, ‘discursive accountability’.
• Think about portfolios: there is too much of a monoculture in IDOs, where every project looks at risk and control separately, as does every country programme, and every organization. Portfolio thinking could change that, seeking a spread of risk across a country programme, an organization, or even between IDOs – the ones with a more secure authorizing environment could take on the messy places, and the ones that have Congress on their backs can stick to bednets, vaccinations or pouring concrete.

Other take-aways?

We had a big discussion when he launched the book in London on whether his findings suggest big IDO should take more aid projects in house, rather than work through contractors (as they increasingly do). His initial comment was that ‘The move to contracting out goes against navigation by judgment – a contract needs to be

fixed, litigable. It’s really hard to litigate relationships’. That isn’t to say it’s impossible to get navigation by judgment in a contract, only more difficult; as Honig put it, ‘It would need non-traditional contracts to crowd in judgment – but that means relations based on trust, not just contract language. How do you know that a contractor will exercise judgment rather than just seek to maximise profits? Options could include looking at its track record, or co-locating donor and contractor staff.’

Honig’s findings are a powerful argument for pursuing Doing Development Differently approaches in messy places like Fragile and Conflict Affected States, where we can now show that they outperform conventional projects. At the launch, Honig said that governance in FCAS is the extreme case – there is almost nothing useful we can count and attribute, either in outputs or outcomes, so navigation by judgement should rule. Doesn’t make it any easier though.