Guest post by Sebastian Kratzer
A few years ago, Alex Douglas from the Centre for Humanitarian Dialogue shared his thoughts on this blog on what aid practitioners could learn from the peace sector’s approach to operating in complex political environments.
But the lessons from the peace world for other aid practitioners can be spun even further. Over the last decade, the peace sector has been developing and adapting Monitoring and Evaluation (M&E) systems and tools to fit their contexts and ways of working. This evolution may hold some insights for the aid community in how to go beyond more traditional, backwards-looking M&E to navigate today’s volatile, interest-based world of politics and aid.
When traditional M&E hits a wall (= reality)
The high levels of complexity, uncertainty and unpredictability in conflict settings make the effectiveness of peace and aid programmes hard(er) to evaluate. Access to stakeholders and data can be extremely difficult. Even when it is possible, ethical concerns abound and the quality of data is so uncertain that often no reliable benchmarking, comparison or analysis may be possible.
Due to the nature of its work, the peacemaking sector faces additional challenges: facilitating dialogue between conflict parties is a highly political and unpredictable venture that relies on trust and relationships.
Under these circumstances, traditional approaches to M&E – with a focus on linear logic, attribution, and causality – have never really been fit-for-purpose in peacemaking, and only sometimes in the aid sector.
An alternative approach to ensuring the value of aid: adaptive M&E
With these limitations in mind, the peacemaking sector had to develop alternative systems and tools. Rather than obsessing about science-grade proof of impact that may never be possible in conflict settings, we have adopted a more comprehensive view on an engagement’s final and interim results, strategy and underpinning logic. This helps practitioners review and adapt their work, while delivering sound ‘proof of concept’ and supporting evidence to donors and the wider public, so that they understand that the work they support is making a valuable contribution to the peaceful resolution of a conflict.
| A three-level adaptive M&E system for peacemaking. Initially developed by Ian Wadley at HD, more on the concept here. |
One example of an adaptive M&E system has been developed by the HD Centre, which looks at both results and logic as determining the value of an activity (shown in the three-level graphic).
From theory to action: implementing adaptive M&E
Having a theory of adaptive M&E is all well and good, but only if we actually manage to go beyond the traditional M&E practices that impose static and linear programming and reporting.
Over the last few years, various peace actors have developed a range of methods and tools to implement adaptive M&E in their engagements. The graphic below, visualizes how we can integrate an adaptive M&E system as part of an (ideal-type) peacemaking approach. This allows us to link our practice to results as well as the tools and methods that are available and capable to measure, evaluate and adapt them.
This creative quest for better tools started with what is now one of the most established tools in the peacemaking sector: peer reviews. These bring together a team with critical peers from inside and outside the organization to reflect on objectives, logic, assumptions, risks and results. They are designed to be low-burden, respond to operational needs, and enable teams to adapt faster and deliver better results in complex environments.
| Results and M&E tools used across the peacemaking sector |
At HD, we have conducted over 60 of these reviews over the last few years. The most recent one dealt with the question of how to bring fresh momentum to a discreet state-level dialogue. The review concluded with a re-orientation of our work towards more sensitive issues: “flashpoints”, i.e. locations or issues with the potential to ignite or escalate conflict between states. Only a few weeks later, our colleagues hosted the next dialogue with an explicit focus on these ‘flashpoints’. Influential figures from participating states welcomed this change of focus to reduce the chances of unintended escalation and were soon discussing the highest-risk locations.
Getting to this point in the peace sector was not easy, nor inevitable – but the result of deliberate action by the peacemaking practitioners to adapt their systems to something more suitable to their reality. In 2014, the HD Centre convened a first donor-practitioner roundtable on the question of how to evaluate peacemaking. The first iterations of this roundtable can be described as a sort of group therapy for both donors and practitioners: airing all that was not well in their relationship. We have since met almost every year, and turned this gathering into the place where the sector quietly develops and presents ideas and consensus on fit-for-purpose systems and tools.
So what’s in it for us?
Looking at the various debates across the development sector, an adaptive M&E approach could make three contributions to those seeking positive changes in the aid sector:
- It functions usually at the project level but can also be applied to the whole-of-organisation level (at HD we have structured our global Results Framework along these lines). A blog about Oxfam’s evaluation practice showed how sustained change is complex and occurs through the interactions of many actors. Adopting an adaptive M&E perspective at a higher level of an organisation may help better analyse, demonstrate, and adapt efforts across an entire portfolio or group of organisations.
- It could support the push for localization and help (slightly) rebalance the power imbalances of the aid sector by empowering the people closer to the changes we’re working for to define what a review (and the reviewed engagement) should focus on.
- It can help align planning and analysis within and between the humanitarian, development and peace sector. It was, for example, integrated into a UK Aid-funded consortium (among other tools) to harness the insights of multiple partner organisations. The Australian Foreign Ministry, DFAT, adapted the methodology to undertake a rapid regional conflict analysis involving their funding partners.
The peace sector’s journey in demonstrating the value of its contribution to peace and stability is far from over, especially as donor and official evaluation policies and guidelines are yet to catch up with these practices. But it may hold a few lessons for the aid sector in delivering and demonstrating the success of aid projects in complex political environments.