Monitoring and Evaluation is broken. Let's really break it.

One of the sad truths that emerged at the Technology Salon on ICTs and M&E was that failure in development is rarely about the project performance, but about winning the next contract. This means that monitoring and evaluation is less about tracking and improving progress towards social change and more about weaving an advertising pitch.

This is not for a lack of frameworks, tools, mapping measurements against a theory of change, or even the need for more real-time data in development. It is about incentives. What is incentivized at the macro level is getting big numbers on the board and nice clean upwardly-trending graph lines. Micro-level incentives for filing reports to fill out the monitoring side of things focus on report filing as a requirement for salary payments or other basic carrot/stick-driven models. Neither of these actually encourage accurate, honest data, yet only with that accurate data can we remotely hope to tweak models and make improvements.

So, let's break monitoring apart from evaluation.

mHealth: Tending the garden of 1000 flowers Jon Tue, 11/30/2010 - 21:11

At #mHS10, we heard funders talking time and time again for letting "1000 flowers bloom" in mHealth pilots, and programs talking about pilots leading to more pilots. This was fine the first few times it came up, but by the last day, the syndromes of the pilot-itis pandemic were clear.

FlowerThis reeks of desperation. The funders are not finding clear winners in their projects, and the various implementers are casting about with local solutions that they either can't or won't scale, trying to find an idea so powerful that it will break through this lock.

We need to focus more energy on innovations which are dealing with core problems in health and in using mobiles for health, and thin out some of these 1000s of flowers. The soil is too fertile for this approach, and the many duplicated, repeated pilots will crowd out new, creative and gamechanging ideas. We need to move past these more basic mhealth applications - reminder messages for drug adherence, pre-natal checkups and so on are great - but simply using a new communications method to address an old problem. Let's replicate and scale those to more sub-sectors and keep them funded, but let's not dwell on them.

On scale -- this is not something that's eay to do. There are many barriers in mobile and in health, from cultural concerns to be dealt with which limit scaling of health projects, to many technical ones inhibiting good mobile projects from being re-implemented in other regions. There are many good meta-solutions to the technical side - open data standards and open source, both praised often at MHS10, are paving the way by creating a variety of tools which can work together.

We need more.

The platforms and networks need to become more open. Projects have been able to thrive where they rely on the lowest common denominators in phones - voice and SMS. Even still, a lack of global short-codes and improved cross-carrier and cross-border functionality hinders scaling. Beyond voice and SMS, it becomes a difficult maze of twisty passages dealing with the various featurephone systems, vendor lock-downs, and even more capable smartphones, which are even more locked down and difficult to get custom applications loaded.

The building blocks are there -- open source tools and open data standards abound; focusing on those is a big first step. Banding projects together, connecting at events like the mHealth summit, and increased best-practice sharing is another. Not being shy about where the real blockers are to scalable solutions is the elephant in the room. Do we need to engage the GSMA and ITU to work on better cross-connection solutions among the many global connectivity providers? Cell phone manufacturers to improve standardized access to their devices? These aren't the low-hanging fruits, but they might be the keys.

From #mHS10 to #mHS11: What we need for next year Jon Tue, 11/23/2010 - 09:43

Herein, a mix of quotidian tasks and big goals for us to prepare for a 2011 mHealth Summit. mHS10 was a great conference, and represents a seachange in the field compared to last year. It had a selection of amazing speakers, lots of academics and implementers, and overall just the right crowd.

Still, there remain some changes I'd love to see for the conference itself, and I'll follow up with some bigger challenges that the mHealth world needs to deal with, based on the themes from this year's conference.

First, keep the music if you must, but add lasers and a fog machine. If you were there, you know what I mean.

My top take-aways from #mHS10 , the mHealth Summit 2010 Jon Wed, 11/10/2010 - 13:53

As you might have guessed from my tongue-in-cheek #mHS10 drinking game (pilot=1 shot, 1k flowers=2, feature phone=3, "going global with sms phone support" = finish bottle), I got a bit tired of "Pilot-itis," which was finally called out as a problem on stage by Christopher Bailey of the WHO during the Wednesday morning plenary.

This pilot-itis was my biggest overall frustration with the discussions and presentations this year - a seemingly endless march of "new" pilot programs around (1) SMS for outreach/awareness (2) SMS and mobile for low-touch scheduled reminders and interaction or (3) Apps for various forms of monitoring. Perhaps it's my relative unfamiliarity with the health field, but do programs do controlled studies every time they plan to release a new paper, or put a PSA ad at bus stops? There is so much that can be done today, with a few hours of hacking, to advance at least #1 and #2 above, settle on a few solutions, and move on to more impactful territory. Take a page from Nike (who have one of the most successful fitness monitoring apps in the wild) and Just Do It.