OK, we’ve had real time evaluations, we’ve done transparency and accountability initiatives, so why not combine the two? The thoroughly
brilliant International Budget Partnership is doing just that, teaming up with academic researchers to follow in real time the ups and downs of four TAIs in Mexico, Brazil, South Africa and Tanzania. Read the case study summaries (only four pages each, with full versions if you want to go deeper), if you can, but below I’ll copy most of the overview blog by IBP research manager Albert van Zyl.
By following the work rather than tidying it all up with a neat but deceitful retrospective evaluation, they record the true messiness of building social contracts between citizens and states: the ups and downs, the almost-giving-up-and-then-winning, the crucial roles of individuals, the importance of scandals and serendipity.
From the overview by Albert van Zyl
‘In the governance field too many learning efforts are based on selective and retrospective documentation of success stories or on politically blind ‘treatments’ parachuted into random contexts. Both approaches undermine the learning potential of governance research. This is why in 2009 the International Budget Partnership and civil society partners Fundar, Ibase, the Public Sector Accountability Monitor, and Haki Elimu took a different approach and gambled on documenting their budget advocacy in real time. This project is part of our ongoing commitment to learning about how, when, and under what circumstances civil society organization’s (CSO) budget advocacy has impact, and how to scale it up. We appointed external research teams to follow specific advocacy campaigns of each of these four organizations over a four year period. Now it’s time to come clean: no airbrushing out serendipity, random events, or failure. Just the stories. As they happened. Think you’re ready?
What did they do? And what happened?
The Public Sector Accountability Monitor (PSAM) monitored, analyzed, and publicized budget and service delivery information produced by the health department of one of the weakest provincial governments in South Africa. While its efforts initially had a limited direct impact on the dismal performance of the provincial government, the national government took note and intervened. As a result, the financial management of the Eastern Cape health department improved substantially after the provincial health minister and head of department were replaced, 31 officials were criminally charged, and a further 800 officials were dismissed.
In Brazil Ibase sought to make the operations of the BNDES (the state-owned development bank with an annual budget four times that of the World Bank) more transparent, and to introduce stronger social and environmental considerations into the bank’s investment criteria. To pursue these objectives, Ibase mobilized a civil society coalition to engage the BNDES. While Ibase had limited impact on its goals, the case study suggests that its efforts have contributed to a greater societal concern for the impact of the BNDES’ operations — opening cracks in the “policy wall” that once shielded the BNDES from public scrutiny.
HakiElimu used innovative methods like their humorous infomercials (“TV spots”) and the mobilization of thousands of Friends of Education to monitor and publicize the finance and delivery of education in Tanzania. HakiElimu demonstrably contributed to improvements in teacher housing, and the scrapping of unpopular programs to fast track teacher training. The case study also produces evidence that HakiElimu’s innovative tactics have rubbed off on other CSOs and even the legislature.
Fundar partnered with grassroots organizations and the Mexican Congress to monitor the budget and service delivery of the Seguro Popular, the national health insurance agency that is supposed to provide health security to the country’s 52 million uninsured. After a frustrating start Fundar assisted the chairperson of the public accounts committee to introduce seven amendments to the 2012 Federal Budget Decree intended to increase the transparency, expenditure control, evaluation, and accountability of the budget of the Seguro Popular.
What’s new about all this?
Mixed results can take sustained effort
All four case studies tell a similar story: while there was limited demonstrable impact on service delivery, there were clearer impacts on the management of government finances and on building the capacity of CSOs and other accountability actors to continue this work. What does this mean? Shall we take the positive view that these intermediate impacts on budgets and civic capacity to demand accountability will eventually lead to future improvements in service delivery? Or shall we be more hard-nosed and say that these campaigns simply failed to achieve their stated goals? Both views are probably right and wrong. As Henry Ford is quoted as saying: “Whether you think you can, or you think you can’t — you’re right.”
As these advocacy campaigns reveal, budget accountability is constructed, it doesn’t “emerge” — there is no magic bullet. Rather, the case studies reveal a fragile and rocky path to impact that needs continuous work, vigilance, and support.
Political misfortune and opportunity
These rocky paths are helped and hindered by political events that often cannot be foreseen at the start of advocacy initiatives. The work of the PSAM was frustrated by the protection that President Thabo Mbeki provided his AIDS “denialist” allies in the Eastern Cape. When he was suddenly replaced by President Jacob Zuma, new national leadership in the health sector took the bait that the PSAM had been dangling for many years. But this is not always the case. In Brazil the imminent election shifted public attention away from the BNDES and threatened to split Ibase’s carefully constructed CSO network along political and tactical lines.
What does all this mean? Apart from making it hard to predict the impact of CSO advocacy campaigns, it is also clear that the ability of CSOs to adapt to shifting political circumstances is at least as important as their ability to front-load their planning through “log frames,” theories of change, or whatever newfangled planning tools they employ.
Playing nicely with others
The trials and tribulations that result from political shifts and realignments also reinforce the importance of other actors in the impact of CSO campaigns. While the shifting sympathies of the national government helped, the PSAM would not have had much impact if consistent media reporting had not kept its analyses on the desks of national decision makers. In a similar way, it was only once Fundar found a willing ally in the chair of the public accounts committee in congress that their concerns were taken into account and carried forward into legislation.
Future efforts to support CSO advocacy for budget accountability, therefore, should not only focus on building the capacity of CSOs but also on building the capacity of other actors in the accountability system – such as the media, legislatures, auditors, and even sympathetic insiders in the executive – and the necessary linkages among them.’
Great stuff. And here’s one final, thought-provoking observation from the Brazil case study, for these results- and success-obsessed times. Studies of economic innovation show that success is often born out of the remnants of a previous failure. Turns out the same is true of NGO work such as TAIs. The Brazil study ends with the thought ‘Someone always has to make the first move. For every instance of successful advocacy, there is usually a less successful case that set the stage first.’