Yesterday I sketched out the theory of change and initial findings on the first four years of work by an extraordinary East African NGO, Twaweza. Today I’ll move on to what some NGO people (but thankfully no-one in Dar es Salaam last week) insist on calling ‘the learnings’ about the flaws and gaps in its original theory of change (described in yesterday’s post).
First, there’s a big ‘black box’ containing Twaweza’s rather large assumption that giving people information (eg about failing education systems), would lead to them taking action to change things. What issues in the black box determine whether this is true or not?
Evan Lieberman (one of Twaweza’s many evaluators, from Princeton University) called this the ’secret sauce’ – the miracle (see much-used cartoon) 
that links information to action. His team had come up with a smart attempt to identify some of the sauce’s ingredients – conditions for a →b:
Do I understand the info? →Is it new info? →Do I care? →Do I think that it is my responsibility to do something about it? →Do I have the skills to make a difference? →Do I have the sense of efficacy to think that my efforts will have an impact? →Are the kinds of actions I am inspired to take different from what I am already doing? →Do I believe my own individual action will have an impact? →Do I expect fellow community members to join me in taking action? Evan argued that only if the answer to all of these is yes, will the black box indeed turn information into action.
Actually it’s worse than that – they missed some pretty big ones (‘do I have the time to do this, on top of everything else?’ ‘Will I run any personal risks if I do this?’) It’s a hell of an intimidating set of conditions and, as was pointed out, the danger is that accountability proponents will just latch onto one of the steps, then wonder why nothing is popping out at the outcome end.
Second, thinking has moved on from Twaweza’s rather Manichean division of politics into the left and right hand sides of informal v formal power (see table), with change to be achieved by the left hand side coming together and demanding action from a nefarious and/or indifferent right hand side. These days, research by groups like the Africa Power and Politics Programme and Matt Andrews argues that both demand side (build the citizens) and supply side (build the state) have failed. What works, they think, is collective problem solving, bringing together citizens, states and anyone else with skin in the game, to build trust and find solutions. People on the ground, like Goreti Nakabugo, Twaweza’s Uganda coordinator, get this: ‘we know we need buy-in from the government, officials, local politicians. We are brokering relationships with them on a daily basis’.
Not only that, but in practice, even differentiating between citizen and state can be problematic – neither category is a monolith, and in some cases, the most active citizens are themselves state employees, members of public trade unions etc.
Third, Twaweza needs to decide whether all citizen action is equally desirable and if not, what kind it wants to promote. Convince a parent that their child’s school is failing and unless they think their protests are likely to get a hearing, they are probably more likely to try and move their kid to a private school than join a social movement to ‘change the system’. Is that a desirable exercise in citizen agency or a systemically disastrous rush for the exit?
Another axis is that of individual v collective action – does Twaweza see them as equal in value or is collective action in some way preferable in terms of building the voice of marginalized groups? Again, not clear from the theory of change. Maybe Twaweza should map the results of its current work on a 2×2 diagram (individual → collective; public → private) and see if it’s happy with the result.
Fourth, envisaging change as taking place in an information ecosystem sounds great, but is it just a metaphor or a real description? According to Rakesh, Twaweza hoped to ‘bend’ the ecosystem towards becoming a driver of citizen action, but ‘to seriously achieve what we had imagined would have required such deep density and high level of coordination and sequencing it would have been akin to playing God.’ The goal is now to try to understand the ecosystem better, and find some less hubristic way of influencing it. A command and control approach was never going to work, so why not 
pull in some actual ecologists to suggest how to influence real-life ecosystems, and see what could carry across into work on accountability?
Fifth, the evaluators came back with some serious questions about irrational optimism over the magic powers of ICT. Eg broadcasting messages to the public via SMS doesn’t work because people only trust the content if they receive it from someone they know. Otherwise they just think ‘spam’ and hit the delete button.
Sixth, an overarching lesson of all this, as so often, is that the system exhibits much more inertia than we think. Things just don’t shift that quickly or easily, despite the best efforts of smart and determined campaigners. But they do (eventually) shift. And presumably some things shift more quickly than others – my guess would be that norms and attitudes (eg over whether it’s worth taking public action to improve your kid’s schooling) evolve much more slowly than public policy. So is it any wonder that the evaluations showed more impact on the latter?
Finally, it is notable that among the partners in the five channels that Twaweza identified as of mass relevance to poor people (mobile phones, media, consumer goods, teachers and religion), only Twaweza’s media partnerships have really worked out. Is that because the media and NGOs are more aligned in terms of language, interests, politics? Would Twaweza need different approaches (change of language, longer timescale) before it can work successfully in the other channels? This is very important – I was a bit disappointed to see all the fine words boil down to doing more and better media work rather than, say, forging real alliances with faith communities.
Tomorrow, I get onto an underlying tension in the Twaweza discussion – the war for Rakesh’s soul (and Twaweza’s future) between the hunger for certainty and the realities of complexity.