This week I found myself on a fun panel at LSE discussing ‘can politics and evidence work together?’ with Mary Kaldor (LSE), Ros Eyben (IDS) and Steven Rood (The Asia Foundation – TAF has a really interesting partnership with LSEto study its use of theories of change). Early last year, I promised to revisit the topic after this blog hosted an epic debate on the politics of evidence between some top DFID people and (in the sceptic corner), Ros Eyben (again) and Chris Roche. Has anything changed since then?
My 15 minutes of fame are summarized in this page of speaking notes: Notes for Thinking and Working Politically LSE panel July 2014
I largely summarized previous pieces on this blog, but added some alliterative categories for some of the obstacles aid agencies face in thinking and working politically, namely:
· The Toolkit Temptation: managing large bureaucracies creates a demand for standardized and simplified procedures (indeed it’s often the first thing hard-pressed staff ask for). Difficult to shift from that to deep engagement with the local context, thinking on your feet and responding to events etc, trying multiple experiments and failing faster etc
· The Planning Preference: Similarly, big aid organizations maintaing internal coherence and external direction by agreeing (at great length) and implementing plans – strategic, operational etc. After years of thrashing out a plan, they are often understandably reluctant to begin all over again because some major event has presented a new window of opportunity
· The Data Delusion: non-scientists in particular are dazzled by numbers, even if they don’t mean much. I still remember my alarm in a meeting of Oxfam’s big cheeses when, after a discussion based on the experience and judgements of senior people with decades working in the development business, someone said ‘right, so much for the opinions – what do the data tell us?’
The other presentations and ensuing discussion (lots of sharp questions, as you’d expect at LSE) drew out some further lessons.
Ros sees actual state of discussion on evidence in DFID and elsewhere as very unreconstructed, with my preferred TWP-compatible approaches still languishing at the bottom of the ‘quality of evidence pyramid’ (see graphic from her slides)
This leads to a major ‘drunk under the streetlamp problem’ of going where the data is, rather than looking at what’s important (see cartoon, not that I’m suggesting MEL people are usually drunk)
Fear of failure is deeply rooted at field level – you can forget all that stuff about celebrating/learning from it, at least when there’s a funder in the room.
The real challenges to TWP are often ‘managerial’ – processes like staff rotation, and the kinds of people you employ.
I’ve been mulling over one issue in particular since then: Mary Kaldor was very opposed to any suggestion that TWP might involve some level of acceptance of the ‘dark arts’ – i.e. working in the shadows, persuading people to do things or talk to others in ways they would not want to become public etc. In contrast, Mary argued for transparency and openness throughout, as the only way to ensure accountability.
Which raises some pretty massive dilemmas: TWP and politics in general often moves forward by opposing forces doing deals that would rapidly collapse if exposed to the light too soon – think of just about any peace negotiation in history, or even just the Northern Ireland one. In NGO world, there’s a reason why advocacy is separate from public campaigning, with different staff, language and tactics. One is based on clear, simplified messages, the other on arm twisting and compromise.
But it must be right to be sceptical of advocacy and policy insiders singing the virtues of secrecy (it’s so easy to be coopted – there’s nothing more intoxicating for an advocacy type than to be the only NGO in the room, get the draft document before all your colleagues etc). Fundamentally, secrecy disempowers anyone who is not in the room, and that usually includes the people we are trying to help.
I wondered if the ‘policy funnel’ – a way to understand how new policies evolve from outlandish ideas to agreed policy – might need to be adapted. We need maximum openness in the early stages, eg trying to get issues onto the official agenda, but have to accept a degree of non-transparency if you get as far as negotiating deals, policies, or cash. But then whatever is agreed needs to be open to public scrutiny (so the funnel becomes more like an egg timer on its side). Mary was entirely unconvinced.
As to the main question, it didn’t feel like much has changed since last year’s FP2P wonkwar on evidence. Maybe there’s been some progress on learning how to measure what matters, rather than just what’s easy to count, and the randomista craze seems to have passed its peak and hopefully RCTs will soon settle down to become one tool among many, rather than the ‘gold standard’ for everything (remember what Keynes said on the real Gold Standard – a ‘barbarous relic‘).
So over to you – do you see progress on reconciling TWP and evidence? And how do you balance TWP and commitment to openness and transparency? Any good examples?
And here’s the rather nice slide that ended Ros’s presentation (don’t suppose Mary will thank me for the panel pic – click to expand and see why)