MEL (monitoring, evaluation and learning) is a lot more interesting than it sounds. Done badly, it can amount to little more than bean-counting to satisfy the donor, of little value to the actual programme or people who are supposed to benefit. But done well, it raises all sorts of really important questions about how the programme/project is designed, early enough to make a difference, then feeds back what is learned (I absolutely refuse to say ‘the learnings’) to improve later iterations of the work. More ME+L then ME+L.
I thought about this when I saw the great new online guide to the MEL of Influencing, from Oxfam. It’s a lot harder to measure the impact of a campaign or a behind-closed-doors bit of advocacy, than more traditional aid activities like vaccines or handing out seeds and tools, which made it even more enticing.
Let’s look at the ‘influencing outcomes’ section, because that’s what interests me most. It starts off with a nice graphic showing potential impacts on people’s lives, from tangible to intangible.
It then moves on to test your grasp of some pretty arcane language (e.g. the difference between outputs and outcomes), and makes some very sensible remarks about why you should be doing all this:
The design is funky (although it’s 100% online – I do wish I could just print it out and read it – sigh). Drop down menus allow you to choose between following it through as a course (v linear, tsk), looking for particular ‘tools’ (e.g. depending on whether you want help with planning, monitoring, evaluating or learning) or want to ‘select a journey’ from ‘building a theory of change’ to ‘influencing outcomes’.
‘Outcomes are usually very important for an MEL System, because they provide early information on whether a project or programme is on course or whether any desired changes are beginning to happen.
If we waited to measure the ultimate impact of the project or programme without bothering to look at the outcomes, it might be to late to realize that we were not in the right track. Any MEL system or process designed to feed into management decision-making needs to assess outcomes on a regular basis.
Avoid M&E system that purely look at what is being delivered on the assumption that if products or services are delivered properly, they will automatically translate into change.
PRIORITIZE OUTCOMES THAT…
You directly influence (rather than indirectly support)
Are important to your goal & mission
Are not too costly to measure
Will produce credible data’
It then moves on to the different variables that can be used to classify or categorize outcomes:
Depending on the Type of Outcome (WHAT is changing?)
Depending on the Actor-stakeholder (WHO is changing?) – some examples for campaigns to change policies
Depending on the Level of the desired change (TO WHAT EXTENT is anything changing?), eg this example on a digital campaign
And that is just section 2.2 – Here’s the full contents list
1 INTRODUCTION
1.1 INFLUENCING
1.2 MEL AT OXFAM
1.3 MEL OF INFLUENCING
2 WHAT IS IT YOU WILL MEASURE?
2.1 THEORY OF CHANGE
2.2 OUTPUTS, OUTCOMES AND IMPACT
2.3 INDICATORS
3 HOW WILL YOU MEASURE?
3.1 SETTING UP A MEL PLAN
3.2 MONITORING; WHEN AND HOW
3.3 EVALUATION: WHEN AND HOW
3.4 LEARNING: WHEN AND HOW
3.5 ROLES AND RESPONSIBILITIES
4 TOOLS AND METHODS
5.A ETHICS OF EVALUATION
5.B GUIDANCE AND TEMPLATES FOR COMISSIONING EXTERNAL EVALUATION
5.C SAFEGUARDING AND MEL
5.D ORIENTATIONS TO DO MONITORING AND EVALUATION OF INFLUENCING PROGRAMS WHERE SCALING UP OR INDIRECT REACH IS RELEVANT
All in all, a real treasure trove for anyone trying to design an influencing exercise, whether an insider/lobby exercise, or a more public protest or media campaign. And as you’d expect from Oxfam, a lot of attention to tackling gender inequalities throughout.
Excellent stuff.
5 ANNEXES