Resilience is one of today’s omnipresent development fuzzwords, applied to individuals, 
communities, businesses, countries, ideas and just about everything else. But how can it best be measured? To plug their new paper on the topic, Oxfam’s measurement wonks Jonathan Lain (left) and Rob Fuller (right) argue with their imaginary non-wonk friend……
So they’ve let the beancounters loose on resilience now. Do we really have to measure everything?
Well, look at it this way: Oxfam projects are focusing more and more on helping people to build their resilience to shocks, stresses and uncertainty. If we want to evaluate the success of these projects and learn from them, then we need to know what outcomes we’re looking for. This means we need a way to think about and to measure resilience.
But can’t you see how resilient people are simply by looking at how well they’ve been able to deal with crises?
Sometimes, yes. Ideally, if we wanted to see whether Oxfam projects have succeeded in building resilience, we would wait and see what happens after people have experienced some actual crises. But often we are trying to evaluate projects that are still ongoing, or that have recently ended. In some cases a crisis may have happened since the end of the project and we can ask people about how they coped: whether they had enough food, whether they lost any livestock or other assets, and so on. But in most cases there has (thankfully) not been a major crisis since the project was carried out. So we have to figure out whether the project that is being implemented now or in the recent past will help people to deal with crises and stresses in the future.
That sounds tricky.
Sure is. The best way we’ve found to deal with this is to identify some characteristics that we think will enable people to cope better with shocks in the future, and then look to see whether households or communities in the project area have those characteristics. Owning assets or having savings are obvious examples. But we also try to think more widely than that: Can people count on support from their neighbours when they need it? Are people aware of climate change and its effects? Do people participate in community groups? Considering these issues has proved useful in many of our previous evaluations of resilience-building projects.
How do you know what the right indicators are?
In a nutshell, we talk to a lot of people who know the local context well, and ask them what characteristics they think 
a resilient household has. Usually this means a combination of doing focus groups with local people, spending lots of time discussing with local Oxfam and partner organisation staff, and reading whatever reports or studies already exist on resilience in the area. This guides us on what questions we should include in our surveys in order to measure resilience effectively.
For example, when we were working in a peri-urban area in Bolivia, many people said that resilience depended mainly on whether or not they had a salaried job. Of course in other places, regular employment is much less of a factor: for example, in Somaliland, people were mainly concerned about having basic access to water, grazing land, and veterinary care – all factors that help to sustain pastoralist and agro-pastoralist livelihoods.
But hold on! If you are coming up with different sets of indicators in each place, you can’t compare the results between countries.
Well, yes and no. We are not trying to create a universal standard measure of resilience. That would be much harder (some might say impossible). So, we cannot say whether people in peri-urban Bolivia are more or less resilient, in general, than people in rural Somaliland. However, what we can say is whether our projects have had more or less impact in each country where we do a survey. When we compare across countries, we effectively normalise or adjust our results according to the average levels of the specific indicators that we used in that context. We can, in fact, compare the percentage change in resilience that our projects brought about in peri-urban Bolivia and rural Somaliland. So even if we aren’t aiming for a universal standard measure, we can at least pull together the results from our evaluations of resilience-building projects.
This approach is all very well for indicators about how people can cope with crises in the short term. But isn’t the idea of resilience bigger than that?
Agreed. The examples we mentioned before could be indicators of people’s ability to deal with crises when they happen. But just as important is whether people can adapt proactively, so they are less vulnerable in the future. That might depend on factors like access to credit, but also on personal, psychological characteristics, such as whether people feel able to take a risk by changing their livelihood or making an investment. It’s harder to find indicators of these more subjective characteristics, but we’ve been experimenting with some different approaches.
Then there’s the issue that resilience is also about transforming the systems that people live in – the village, the region, the country, or even the world… If our projects can change the way these systems work – for example, by advocating for policy change or protecting the environment – we can limit the stream of shocks and stresses that people face. In many of our previous evaluations, we have tried to measure resilience simply by collecting large-N data through household surveys. But if resilience is also about stuff that happens at the level of the system – the village, the region, and so on – then it also makes sense to collect small-N data at the system-level too. This is exactly what we have been trying to do. For an example of the kind of rigorous qualitative techniques we want to incorporate into framework for measuring resilience, see this previous blog post on ‘Process Tracing’ by Claire Hutchings.
This is all work in progress, and we are constantly feeling our way and trying to improve our methods for thinking about and measuring resilience. If you would like more detail, please read our new paper. And, if you’d like a deeper dive into Oxfam’s resilience Effectiveness Review evaluations, including the newly published set, they can be accessed here.