Jane Lonsdale reflects on the lessons from an ‘effectiveness review’ of a Myanmar project 18 
months after it ended. For the nerds among you, an accompanying post on the nuts and bolts of the effectiveness review has just gone up on the ‘real geek’ blog
We have just finished a review of Oxfam’s work in Myanmar’s central Dry Zone. This was designed some 6 years ago – a lifetime in Myanmar, back when the country was far more closed, and we were trying to disguise governance work as a resilient livelihoods project. Our intentions were to help communities prepare to face future potential upheaval, including:
- Political –it was very uncertain what direction Myanmar would take back in 2010,
- Economic – such as sharp increases in agriculture inputs or falls in agricultural prices, or
- Climatic- droughts or floods in the case of the Dry Zone – which the area is annually at risk of.
To do this we focused on building durable community structures, knowledge, and jointly exploring existing and new ways to cope with potential uncertainty. The activities included community development, Participatory Community Vulnerability Assessments and planning, a revolving fund as a way to support the most vulnerable in the community to handle the hungry season, and support on domestic and agricultural water, collective buying and selling, and cotton and beans growth and trade as the two value chains that could work best for women.
This was our working definition of resilience, and what we thought it would take to increase it in this context.
The study looks at how many of the Member Organisations (a version of Community Based Organisations) that we helped to establish were still operating, to what degree, and what made them continue. We were working in areas where no structured community organizations had previously existed, with little NGO presence at the time, so Oxfam had to use a mix of direct implementation and working with NGO partners, where they existed.
What did the review find? Some success, some failure, and a lot of interesting lessons. Of the MOs researched, 18 months on 62% were ‘hopeful or maturing’, and 38% ‘struggling’; none were completely dysfunctional or 100% effective. What I found most interesting was the strong correlation between good MO performance, and strong livelihood and resilience improvements (and vice versa – weak MOs correlated with weak improvements), supporting our hypothesis that
The inevitable theory of change diagram
capable and well functioning MOs can make a difference in current and future livelihood conditions. That backs the case for investing in social capital as a route to tangible improvements in people’s lives and livelihoods. This increased resilience came even despite severe floods in 2015. The big takeaways for me were:
- Trust emerged as a key factor in the success or failure for the MOs. This speaks to the findings on much of our governance work, in both conflict and non-conflict affected areas. In this case activities on trust were not an explicit focus, but they certainly will be in all future projects.
- Better access to government services correlates with the areas where there are stronger MOs. But which comes first? Did the MOs use collective power to secure better services, or were
they stronger in the first place because they had stronger social capital from better education, healthcare, nutrition etc? Or are there influential individuals in those villages that are able to both secure government services and build a strong MO? Possibly a combination of factors at play, but it begs the question, what do we do about building MOs in areas where there are little or no services? Is it just more intense support that is needed or a new design altogether, do we in fact have to deliver those services for a while before we get into longer term development work? Or will that cause damage by interfering with the model where the government provides services and the community hold them to account over those services? - A lack of enabling environment – that is, a local government that is willing and able to engage with communities, was a key constraint – the government was very closed at the time in this region. This meant that all our efforts went into building the demand side (organizing people to make demands on government). But when the political opening came along, the project benefited from a change in policy when local development funds started to flow towards the project end – and the MOs were ready with their resilience action plans and in many cases able to secure funds for their priorities. So what do we do in closed areas such as this? Just invest in the demand side ready for when things open up to engage with the government? Or invest time in trying to get a breakthrough? How to do a cost benefit analysis on such a dilemma?
- Women in leadership increased after the exit, to more than 40% 18 months on, and only 13% of those asked in the community expressed distrust in women’s leadership capacity. These are decent results, but it’s difficult to work out why this happened, even with this detailed level of review. Was it down to good gender programming (we certainly invested a lot of time and energy in this), or did external factors such as the success of Aung San Suu Kyi as a female role model have an effect?
- MO membership massively increased after the project finished, and the MOs came together and in some cases, secured donor funding. Why did people continue to join and decide to collaborate? Was it a coping mechanism once the NGO support stopped? Is the project closure in fact a good thing, exiting before dependency sets in?
- What would it take to move the Hopeful MOs to a stage of Maturing MOs? (Bearing in mind these typologies had pretty high standards in this review). Is it more time, better design, or just the level of enabling environment or access to services that would make the difference? Or does it all come down to individuals with leadership and social capital at the village level?
As ever, community development done well is resource intensive and it is difficult to demonstrate cost effectiveness. But if you’re going to invest in such intense processes, the key is to get that initial investment as right as possible, demonstrate the value for money where you can – and build in space for adaptation as you ‘learn by doing’ and/or conditions change. The review has given us tonnes of insights for future programme design on success factors and risk factors in local governance and resilience building. The question now is how to institutionalize such knowledge so that learning reviews do indeed feed into the future?