Do Aid and Development need their own TripAdvisor feedback system?

I’ve been thinking about TripAdvisor recently, as a model of fast, crowdsourced feedback which highlights rubbish hotels and

restaurants, and creates pressure for them to shape up.

There’s plenty of rubbish performance in the aid and development sector, but our feedback loops are mainly limited to conversations in corridors and the occasional email. So what would be your top candidates for a developmental TripAdvisor?

Let’s start with aid organisations themselves. My colleague James Whitehead asks: “Is there space for ‘Rate My Aid’? In a humanitarian crisis the affected populations are often over-surveyed yet have very little voice in the services they receive. Feedback from communities could be cross-referenced against data on back donors to create a leaderboard of best UN agencies, best INGOs, best local NGOs, and worst… If I was a donor, I might look to the leaderboard before I looked to the project report – to what beneficiaries say about the service rather than what the service deliverer says about it.”

Aid organisations could rate each other too. Imagine a system where grant recipients could feed back anonymously on the behaviour of their donors; do they listen to you? Make unfeasible reporting demands? And vice versa – did money go missing? How big were the over-runs? Which ones really delivered exciting programmes?

Another topic could be the corporations who are looking for NGO/official aid agency partners – some of this is a genuine interest in development, some is PR spin. It would be great if you could just click on the feedback for any given company and find out who is serious, and which ones are just Uber/UN Women PR disasters in the making.

But why stop at organisations? What about individuals? Here’s where it gets a little trickier.

What about rude aid workers? When humanitarian agencies and their partners are filling in a gap in state provision, a government could easily insist that all aid workers wear a unique identifying number like police officers. Combined with clear crowdsourcing you could easily see which agencies treated people with dignity and which staff might need further training. The days of humanitarian agencies acting as unaccountable non state actors could be numbered…

Consultants: they charge the earth, often for substandard work. How do we find out in advance which ones are good, and which ones just endlessly cutting and pasting the conclusions from their last report?

Journalists: they phone you up, suck your brains dry in search of new ideas for their next piece. Which of them actually write

anything as a result, or credit you or your organization when they do so?

Peer Reviewers for research proposals: Half of them don’t deliver, or use the opportunity for cathartic venting about their hobby horses, rather than actually reviewing the proposal.

Civil servants: which ones really listen and interact, which go through the motions?

Campaigners and Advocates: Do Government or World Bank staff get to comment on how they feel about the work of different aid agency lobbyists?

Some design issues:

There is likely to be a huge difference between feedback from targets and from peers – any way to differentiate?

Everyone will try and game the system (‘sure I’ll take that meeting, but you have to give me a good review’). I wonder what TripAdviser does to avoid that?

The dangers of blacklists: at the very least you would need an appeal procedure against unfair reviews. Would that make it unfeasibly expensive and bureaucratic?

I sent these thoughts to a couple of IT accountability gurus: Martin Tisne at Omidyar asked some good questions:

‘Is it 2-way or 1-way? Who is the ‘client’ for this type of platform? With TripAdvisor, the clients are the paying customers of the hotels – i.e. it’s not 2-way, the hotel does not get to exercise much choice in terms of the customers it takes on or not, no black listing etc. Uber and the like purport to have a 2-way system with customers rating cab drivers and cab drivers rating customers but I suspect that on the whole it’s the cab drivers who are really being assessed rather than the other way around. So what are the experiences out there we can learn from that are genuinely two-way and what have they learned about [dis-]incentives to publish and how to deal with conflicts of interest?’

While Tom Steinberg added some sage advice, based on his experience of setting up lots of such feedback sites in the UK:

‘The art to succeeding here is to start by choosing just one kind of review of one very carefully chosen kind of service – one that has been chosen to minimise the perverse incentives and so on. Try to get that very narrow service working and then move on from there.’

What do you think?

An edited version of this piece appears today on the Guardian Development Professionals blog.