In defence of empiricism – what social investing can learn from science
You may not have come across the term ‘empiricism’ before, and if you have, you may wonder what it has to do with impact investment.
The empirical method was first used by a group of ancient Greek medical practitioners who, frustrated by medical approaches rooted in dogma and superstition, began to promote practices that were based on observation. These early medical practitioners wanted to understand symptoms and treatments in a way that would not rely on religion or superstition but rather would deal with observable facts. This shift eventually led to modern medicine as we know it today, where medical breakthroughs and new treatments come from rigorous research and experimentation.
The empirical method involves setting out a well-defined and focussed question, for example, does a new form of psychotherapy lead to better mental health? Core to the empirical method is the use of rigorous research methods—this means using measures that have been tested and validated, using control groups that have either been randomised or have been matched to the group under observation and using statistics to determine true effects from chance.
Building a bigger picture
Most importantly, it combines this research into a theory for a certain field. This means that while individual studies might be niche in focus, they add to a greater body of research that can help separate fact from myth in different disciplines. Because there are consistent methods, measures and statistics, studies can be combined to answer important research questions. For example, does a new drug lead to improved patient outcomes or does a certain education intervention lead to improved grades?
These studies, known as systemic reviews or meta-analysis, can tell us whether yes, there is an overall meaningful effect or no, there is no effect as a result of a new intervention. This is extremely helpful as it means that society can benefit from interventions that are known to have a real positive benefit.
Sounds great, right? Other disciplines certainly seem to think so: modern science can trace its roots to the scientific revolution of the 16th and 17th century, sciences like psychology and education have been embracing the empirical approach since the beginning of the 20th century and even the private sector has been harnessing the power of data and experimentation (Facebook’s recent mood experiment may have been controversial but it used good science).
What’s this got to do with impact investment?
So this begs the question: Can impact investing also benefit from experimental approaches to research? We would certainly like to see this happen. Nesta Impact Investing’s approach to impact measurement promotes the use of rigorous methods, control groups, randomisation and replication.
However, it seems that we have a long way to go before this sector embraces such an approach. The June 2014 survey by GIIN and JP Morgan found that while 95% of impact investors use standard impact metrics, only 8% consider it important to gather data on the efficacy of investments, essentially suggesting that rigorous evaluation of whether an investment truly has a positive impact is sorely lacking.
This is a concern for many reasons. Firstly, it throws doubt on whether impact investors can truly talk about the impact we are having. Secondly, it means that across the sector, many players are using wholly different and mostly self-developed methods to understand social impact. Finally, we will not be able to combine our research and evaluation work to answer big questions like whether a new product or service has had a positive effect.
Right now, impact measurement means a lot of different things to a lot of people in this space. I would argue for less diversity and more embracing of an approach that has been tried and tested over the centuries—rigorous science and the empirical method.
By Eibhlin Ni Ogain – Nesta Impact Investments