Opening Pandora’s Evaluation Box

“Revolutions in science have often been preceded by revolutions in measurement.” Based on the premise from Sinan Aral of the MIT Sloan School of Management

Jason Saul presented to a full-house as part of the MaRS Global Leaders series in April 2016 on his latest venture – the Impact Genome Project (IGP). A public-private partnership to code and quantify the “genes” of what works in social science. The audio of the presentation can be found below.

If you spend 5 minutes in the social impact sector you are sure to be asked, how do you know you are making a difference?

Screen Shot 2016-04-26 at 6.07.08 PM

Jason Saul and Nolan Gasser present the Impact Genome Project at the Skoll Forum in 2014

Jason and his colleagues at Mission Measurement have been tackling this question by taking us from the current state: we are spending $400 billion to achieve social outcomes without any standard way to accurately measure ROI; the evaluation industry is in disarray; evidence is unstructured and unintelligible; and yet evidence is growing exponentially – it is just not readily accessible. We have no common language; no benchmarks that allows us to compare social programs; and ultimately no predictive data meaning we can’t forecast before we invest. This is what Mission Measurement calls the black box problem.

Yet other sectors have predictive data and use it to increase their impact: think credit scores, the human genome or even Netflix. The music industry has cracked this code with Pandora and their Music Genome Project, the original inspiration for the Impact Genome Project. Jason approached Nolan Gasser, the architect of the Music Genome Project, and together they embarked on a journey of discovery asking one question:  Can we not do for social programs what Pandora did for music?

Screen Shot 2016-04-26 at 6.04.06 PM

You can watch a short documentary on the story of Pandora and Nolan Gasser on FiveThirtyEight

As it stands the Impact Genome Project comprises 11 total genomes: education; economic development; public health; youth development; international development; human services; criminal justice; sustainability & environment; science & technology; arts; and culture & identity. With 132 common outcomes. The goal of IGP is to produce new benchmarks such as efficacy rate; expected outcomes; and cost per outcome. It is an open data project with advanced analytics available via subscription.

The IGP intends to create a more level playing field by:

  • Democratizing evaluation
  • Replacing guessing with data
  • Learning systematically across the sector
  • Unleashing innovation and creating twice the impact with half the cost

It is an audacious goal and yet the future is here. The UK government is already moving to pay for outcomes and have created a What Works Centres, a network of centres to “support more effective and efficient services across the public sector at national and local levels.” Our own governments are not far behind with the Centre of Excellence for Evidence-based Decision Making Support at the Government of Ontario, which was part of Minister Deb Matthews’ mandate letter.

The UK government released a report in 2014 providing an overview of the What Works Centres.

The UK government released a report in 2014 providing an overview of the What Works Centres.

Why Canada? We have supportive genetic infrastructure:

  • Un-entrenched philanthropic institutions
  • Integrated and collaborative philanthropic sector
  • Government prioritizing evidence and value for money
  • Institutions willing to lead
  • Access to top talent/academic institutions
  • Systems-thinking expertise

We are interested in what you think. Does this seem like a way to get ahead of the inevitable move to pay for outcomes? Can we work with funders to make this approach the standard, not the only way forward but one that is “directionally correct”? What are your concerns, if any?

Please let us know and help us determine how we can get to a better place around demonstrating our impact in a world that needs us to use all our talents to tackling our complex challenges.

Print Friendly, PDF & Email
Bookmark and Share
Allyson Hewitt About Allyson Hewitt

Allyson Hewitt is the J.W. McConnell Family Foundation's Senior Fellow in Social Innovation at MaRS Discovery District

Speak Your Mind