Category: Evidence

How can we invest for profit and social impact? How can we understand the impact of a diverse portfolio? How can we help to embed social impact into commercial businesses? Our report Setting Our Sights: A strategy for maximising social impact describes how Nesta Impact Investments is tackling these big questions at the heart of impact investing. It describes how we go about selecting investments we believe will change lives and our approach to managing for impact after investment.

 

The updates to our approach are based on five lessons from five years of impact investing:

1. The importance of alignment While the portfolio is still young, our strongest investments to date have demonstrated alignment between commercial success and social impact. We have been particularly successful with businesses where impact is core to their commercial strategy. For example, Oomph! use their evidence of impact on older people’s health and wellbeing as part of their sales strategy. As a consequence, we have introduced alignment as a specific requirement for investment.

2. Impact risk has many dimensions Our previous approach defined impact risk as the standard of evidence provided by that venture. While this is an important factor in whether estimated impact is realised, we have learned in the last five years that there are other factors we can identify at the point of investment that may raise or lower the risk to achieving social impact. As a result, we have broadened our definition of risk to reflect the importance of the skills and attitudes of the leadership team and the logic underpinning how the product or service will lead to impact.

3. Setting expectations We have found that codevelopment of an impact plan is a crucial time for setting mutual expectations about the time and effort required to measure impact. Where it has worked well, agreeing the impact plan has been an opportunity to ensure that the venture is well resourced to deliver on impact measurement and that timelines for data have been sensitive to business milestones. We are committed to ensuring that a collaborative and realistic impact plan is set out before every investment.

4. The challenge of maintaining focus on impact Our highly standardised approach to a very narrow impact assessment framework for early investments in NII meant that we did not always strike the right balance of flexibility to changing business needs with maintaining a focus on impact. As a result of this inflexibility some ventures became increasingly distanced from their impact plans to a point where measurement and engagement ceased to be relevant. Turning the situation around could then be extremely challenging as the venture, by then, had often lost its focus on measuring impact. In future impact investment funds NII will strive to maintain a robust, but flexible approach to impact plans so that wherever a portfolio company starts to deviate from its impact plan, we work with the management team to develop a strategy for getting back on track or adjusting the plan in the same quarter. We will ensure that impact conversations continue to be held quarterly, no matter what the situation.

5. Impact at the investment committee In the past we made sure that there was social impact expertise on the Investment Committee. Earlier in the life of the fund this expertise was concentrated in one person who often had to ‘fly the flag’ for impact alone. As the fund has progressed we have taken steps to ensure that all committee members feel equipped to participate in substantive conversations on social impact by making our impact assessment more transparent and consistent, highlighting particular areas for discussion. In future impact investment funds, we will continue to review the dynamic of conversations in the committee to maintain proper oversight and accountability for social impact.

 

The framework presented in the report seeks to test many unproven assumptions. As we pilot this new approach we welcome thoughts from mission-led businesses, other impact investors and asset owners for how we can strengthen our approach. The questions we will be asking ourselves are set out in each section. If you have reflections on these questions or think there are other questions we should be asking please get in touch.

This blog originally published on the Nesta website http://www.nesta.org.uk/blog/new-impact-approach-nesta-impact-investments

Impact investing faces an existential threat and we are sleep-walking into it. Low accountability for impact is leaving us wide open to attack at the first sign of trouble.

 

The experience of the charity sector should certainly give us pause. Charities have been caught on the hop – headlines decrying high salaries or incompetence (Kids Company, Marie Stopes) abound – and cannot effectively be countered with a compelling account of what the charities achieve with their money. A musical about the decline of Kids’ Company goes on stage this summer at the Donmar Warehouse, we are both appalled and fascinated by these examples.

 

In the court of public opinion, it is not hard for the government to start cutting grants that sound wasteful when the defense has no compelling evidence as happened earlier this month.

 

Public trust in charities is going downhill but it is unthinkable that charities as we know them will disappear. The abiding sense that these are organisations with the essential purpose of doing good remains. Charities may sometimes screw up, hire the wrong people, fail to meet professional standards but surely these are bad decisions made with good intentions? Why else would charities exist? And aren’t they regulated? And doesn’t your aunt volunteer for a local charity? They can’t be all bad.

 

The more cynical among us may not be so easily convinced but I find it unlikely that the charity sector is going to suffer anything more than an unpleasant kicking.

 

By comparison, impact investment has none of this automatic, occasionally misplaced, trust. We are not rooted in the community, we do not have hundreds of years of history, we even *gasp* try to make money.

 

I can tell you that, like many impact investors, NIM does indeed take impact very seriously, as seriously as financial returns. I can describe to you our impact assessment process, our requirement for companies to enshrine social impact in their articles, our divestment policy for when things go wrong. But we do these things because we want to, because we believe this maintains our focus on impact. No one makes us do this. And to me that is too fragile, I want to be held to account for the impact I claim we achieve.

 

There is currently a surge of interest in impact investing with everyone from big banks to boutique wealth managers offering impact investment products. This is the moment when we can capitalise on asset owners’ willingness to take a bit more risk in return for doing a bit more good. Resources are growing and we are forgiven many of our weaknesses on the basis of youth. This gives us a freedom to innovate to develop our field, but also a great responsibility to do so in a way that will build a solid, differentiated and defensible foundation for future investors and managers to build upon.  This is a moment must not be wasted.

 

The Impact Management Project coordinated by Bridges Ventures, that we are supporting and participating in, will hopefully make 2017 a year when we take a huge leap towards an agreed convention for understanding and expressing impact. We are also excited to be working with organisations driving the development of the Impact Investment market such as Big Society Capital and the European Investment Fund who take this problem seriously, and whose influence is felt well beyond their own portfolios.

 

From what I have seen I think we can and will build a robust accountability framework for impact investors. The will is there. But I worry sometimes that the urgency to establish such a framework is not felt keenly enough across the sector. For impact investing the cost of failure really could be painfully high.

 

Where robust accountability structures don’t exist or are ignored there is a real risk that there will be scandals. Impact investing, doesn’t yet have these structures, yet like any form of investing, involves risk and so we are vulnerable to Kids Company moments and crises of confidence. Moments when the sceptics jump at the chance to say ‘we told you this was all a smokescreen for making more money’.

 

We must be ready. Ready with robust evidence and ready with the accountability mechanisms that mean people don’t have to take what we say on faith. If we don’t, then then we may disappear as an industry altogether. Another bright idea that turned out to be too good to be true.

This article is part of a series of blogs offering our tops tips on surviving the process of raising impact investment. Investment raising is difficult and time consuming, and the process can seem daunting if it’s your first time.  We have written this series of blogs as a way to share some insight in to what impact investors are looking for.

---

 

Raising early stage investment is difficult, there is no doubt about that.  In Silicon Valley one in every 100 business plans read by a VC receives investment.  For a startup looking to raise investment, this must be quite a scary statistic.

Similarly, at Nesta Impact Investments, we have received hundreds of applications for investment and only a small percentage meets our interest – 4% to be exact!  Since launching Nesta Impact Investments in November 2012 we have read approximately 300 business plans, probably met with about 25% of these, and pitched 12 investment opportunities to our investment committee.

So with only a 1-4% chance of securing investment and the pitch being your way in (no pressure!), here are our top 5 tips for pitching to an investor:

1. Keep it simple. 

Try to present your venture in a clear and concise way.  The next step is for us to pitch the company on your behalf – so make it easy for us.

2. People matter

In early stage investments we are backing people to deliver on their plan.  Some ventures we look at don’t have a track record – but the founders will.  Tell us about you and your co-founders upfront – don’t leave this till the last slide.

3. Show the passion

Don’t hold back on showing us your passion for what you’re doing.  If you’re not excited about it, the chances are we won’t be either.  Explain the need for the product you are building and why it will exist far into the future.

4. Be concise. 

We want to hear your pitch and have time afterwards to ask questions that will really help us understand you and your business.  If you have a one hour meeting, then a 15-20 minute pitch is ideal, but  a pitch that lasts 30-45 minutes will limit the time we have – and probably our concentration span.

5. Do your homework.  

Know what the investor you are pitching to is looking for and tailor your pitch.  As an impact investor we prioritise social impact, so we look for compelling pitches that convince us not only that an entrepreneur can create a thriving business but also that he or she can transform millions of lives.

Pitching can seem scary, but these five tips will hopefully calm your nerves!

---

By Katie Mountain – Nesta Impact Investments 

This article is part of a series of blogs offering our tops tips on surviving the process of raising impact investment. Investment raising is difficult and time consuming, and the process can seem daunting if it’s your first time.  We have written this series of blogs as a way to share some insight in to what impact investors are looking for.

---

 

The good news is that as a startup, you probably already have most of what it takes to develop a good impact plan. Spotting a gap in the market and developing an innovation that plugs that gap requires a skill-set that comes in handy when thinking about impact.

You start with asking the right questions. For example: Who will benefit? How will they benefit and why is this important?

Such questions can lead you to developing your Theory of Change: a visual map of why and how your enterprise will make a positive social impact. This roadmap will then guide you to choosing the right impact metrics for your venture.

Going through this process not only gets impact investors like Nesta Impact Investments excited but also helps you to focus on the social mission at the heart of what you do. So our top tips for getting started on an impact plan are:

1. Identify what social issue you are trying to address

What are the factors behind this problem and why will your venture help to tackle these factors?

2. Outline why your innovation will make a difference

Are you duplicating what is already out there or are you addressing a genuine gap? Do you have any research or evidence showing that your product or service can help tackle the issue you are addressing?

3. Map out how your innovation will have an impact

Try to map out the short- and long-term impact of your product or service and why it links to your final vision. A good tool for this is a theory of change which helps you to connect what you do on a day to day basis with the overall change you’re trying to make.

4. Choose the impact metrics you will use

Qualitative data like case studies or observation are good sources for a rich and nuanced picture on whether change has happened. Quantitative research is useful to understand effects on larger groups of people. Questionnaires, or data on numbers of people qualifying from a course, for example, can all be used to gather information quantitatively. Inspiring Impact has a great hub to help you choose different types of impact metrics.

5. Work out how you will use your impact data to learn and improve

Make sure you learn from your data and make changes to what you do as a result. Ultimately, impact measurement is about understanding the impact you are having and improving your product or service as a result. If the data you collect is not helping you do this, you need to go back to the drawing board and develop an approach that will.

There are many resources out there that can help you develop your impact plan. ThinkNPC’s website has a number of reports on Theory of Change and impact measurement; the Inspiring Impact website has a good collection of tools; and the G8 report on impact measurement in social investment outlines a best practice guide for impact measurement in social investment.

But remember, the best place to start is asking the right questions.

---

By Eibhlin Ni Ogain – Nesta Impact Investments

 

You may not have come across the term ‘empiricism’ before, and if you have, you may wonder what it has to do with impact investment.

The empirical method was first used by a group of ancient Greek medical practitioners who, frustrated by medical approaches rooted in dogma and superstition, began to promote practices that were based on observation. These early medical practitioners wanted to understand symptoms and treatments in a way that would not rely on religion or superstition but rather would deal with observable facts. This shift eventually led to modern medicine as we know it today, where medical breakthroughs and new treatments come from rigorous research and experimentation.

The empirical method involves setting out a well-defined and focussed question, for example, does a new form of psychotherapy lead to better mental health? Core to the empirical method is the use of rigorous research methods—this means using measures that have been tested and validated, using control groups that have either been randomised or have been matched to the group under observation and using statistics to determine true effects from chance.

Building a bigger picture

Most importantly, it combines this research into a theory for a certain field. This means that while individual studies might be niche in focus, they add to a greater body of research that can help separate fact from myth in different disciplines. Because there are consistent methods, measures and statistics, studies can be combined to answer important research questions. For example, does a new drug lead to improved patient outcomes or does a certain education intervention lead to improved grades?

These studies, known as systemic reviews or meta-analysis, can tell us whether yes, there is an overall meaningful effect or no, there is no effect as a result of a new intervention. This is extremely helpful as it means that society can benefit from interventions that are known to have a real positive benefit.

Sounds great, right? Other disciplines certainly seem to think so: modern science can trace its roots to the scientific revolution of the 16^th and 17^th century, sciences like psychology and education have been embracing the empirical approach since the beginning of the 20^th century and even the private sector has been harnessing the power of data and experimentation (Facebook’s recent mood experiment may have been controversial but it used good science).

What’s this got to do with impact investment?

So this begs the question: Can impact investing also benefit from experimental approaches to research? We would certainly like to see this happen. Nesta Impact Investing’s approach to impact measurement promotes the use of rigorous methods, control groups, randomisation and replication.

However, it seems that we have a long way to go before this sector embraces such an approach. The June 2014 survey by GIIN and JP Morgan found that while 95% of impact investors use standard impact metrics, only 8% consider it important to gather data on the efficacy of investments, essentially suggesting that rigorous evaluation of whether an investment truly has a positive impact is sorely lacking.

This is a concern for many reasons. Firstly, it throws doubt on whether impact investors can truly talk about the impact we are having. Secondly, it means that across the sector, many players are using wholly different and mostly self-developed methods to understand social impact. Finally, we will not be able to combine our research and evaluation work to answer big questions like whether a new product or service has had a positive effect.

Right now, impact measurement means a lot of different things to a lot of people in this space. I would argue for less diversity and more embracing of an approach that has been tried and tested over the centuries—rigorous science and the empirical method.

---

By Eibhlin Ni Ogain – Nesta Impact Investments

You’ve come up with a great idea, sorted out your business model, built a strong team, dealt with cash flow and all the other aspects of setting up your own business. And then you discover that you also need to think about whether you are having a positive social impact.

For social start-ups, measuring impact can often be the last thing on a long list of essential to do’s. But it is an important thing to get right. If social start-ups stand a chance at success they need to integrate the social side of what they do with their core business model and prove their impact. Measuring your social impact should be dealt with in the same way your business understands and measures its finances. And the good news is that there are processes and methods you can use to do this.

So where should you start?

Step one: where do you want to get to?

The starting point for any social venture is to understand their final vision for impact and how they are going to get there. A good tool for mapping out this process is a theory of change. This will allow you to connect what you do on a day to day basis with the overall change you’re trying to make.

Step two: Decide on the type of evidence you need

The type of evidence you collect through measurement can range from less to more robust. Early stage ventures may not need to collect at the robust end of that range as they will probably refine and develop what they do as they grow and change. At a minimum, you should have a clear articulation of how you deliver social impact. Nesta Impact Investments has developed Standards of Evidence for Impact Investing to help investees think about these different levels of evidence.

Step three: Choose the tools and methods you will use

Once you have developed a theory of change, you need evidence. The best way of doing this is to collect information on whether change is happening. This can be done in lots of different ways. For example, when we talk to people or observe them, we are collecting information about their views and behaviours. This type of qualitative research can be used to get very rich information on whether change has happened. It is not good, however, for getting an idea of whether everyone you worked with saw similar effects or understanding how big this change was. To answer these kinds of questions, quantitative research can be used. Questionnaires, or data on numbers of people qualifying from a course, for example, can all be used to gather information quantitatively. The diagram below shows the different types of tools out there for capturing information. Inspiring Impact also has a great new hub to help with this.

Step four: Think about when you will measure and who you will measure with

Timing is important when it comes to measuring change. Think about when it is reasonable to expect change to happen—this could be immediately, or three, six months or one year down the line. Who you measure with is important as you may not want to gather information from everyone—this may be because it will be too resource-intensive or interferes with your service delivery. So, you may consider gathering information from a smaller sample. Generally a sample of fifty will be necessary to see change but the Survey System website can help you decide.

Step five: Use the results

Finally, as a start-up you will more than likely develop and refine your product—and as you do, your understanding of your impact will change. Just as the lean start-up approach argues continuous refinement of your product based on customer feedback, impact measurement should inform you about what is, and is not, leading to change. You’ll need to adapt your Theory of Change to reflect this and review how you measure your impact.

Remember that impact measurement is about understanding whether you are having a positive impact on the issue you are addressing and improving your product or service as a result. Ultimately, if the data you collect is not helping you do this, you need to go back to the drawing board and develop an approach that will.

---

By Eibhlín Ní Ógáin – Nesta Impact Investments
This blog was originally published on Pioneers Post 
Read the original blog

Nesta has long been interested in ascertaining the impact of its work, not only to gauge where we are being effective, but to understand how effective we are being at promoting our broad pro-innovation agenda (from policy suggestions, grant-funded trials, to our investment activity).

At Nesta Impact Investments our aim is to source high impact solutions to some of the UK biggest challenges and through investment and support take those solutions to scale.  To achieve this, the first step is to understand the impact each venture has on our target outcomes.

Measuring the impact of a product or service isn’t something that’s exclusive to the social sector.  Organisations from every sector have a vested interest in understanding the user experience of their product or service. Added to which, end users want to know that the product or service they are receiving will deliver against its promise – does it really do what is says on the tin? 

To offer a seasonal example, would you buy sun tan lotion without a clear statement about the UV protection benefits it offered?  If there was no proof of its effectiveness would you spend your money on it and use it in this sunshine?  Probably not, I’d guess. When making such a purchase decision, we want to know that the stated benefits have been tested, proven and verified.

To take a more relevant example, at least from the perspective of our investment activity, if a school is looking to buy an educational product, it will at the very least want to believe that this product will have some benefit to the speed/depth/breadth or effectiveness of learning. If the provider of this product can go a step further and actually demonstrate and verify how much faster/deeper/broader the learning experience will be, then it is our belief that, with all other things being equal, the school will be more inclined to purchase a product with evidence of impact.

There are many other such examples from all sectors of the economy. If an organisation can isolate and demonstrate the effectiveness of its offering, it will be in a stronger position to win more business and grow.  We would argue that this is the most valuable metric an organisation can measure.

At Nesta Impact Investments we are committed to understanding the impact performance of each venture we work with, both in terms of the numbers of people benefiting from a product or service and the effect of the product on people.  We are clearly not alone in harbouring a deep rooted interest in the effectiveness of the interventions we are financing.  The social sector has been wrestling for some time with attempts to increase the practice of measuring and reporting of social impact, and the Inspiring Impact campaign is a great example of the real momentum that is now building amongst charities on this topic. Government is increasing its focus on the role of impact, and measurement of it, as the constrained public purse seeks to get better outcomes for less money – Payment By Results (PBR) contracting is a high profile policy example of the public sector’s need to define and measure outcomes.

The commercial sector has created whole industries to support its efforts to create product/service differentiation, strong user experiences and brand loyalty – all based on the need to set, meet and ideally exceed customer’s expectations of their offering. The advertising and marketing industry is based on the idea of better understanding your customer’s needs and wants and then measuring how you have got-on delivering against these expectations. Without gauging how your customers interact with and use your product/service, you will struggle to scale-up and optimise the offering.

In our view, to successfully grow organisations you need to evaluate the effectiveness of any end-user offering whether in the social, public or private sector.  Measuring impact is something we will do with all of the organisations we invest in, through our own Standards of Evidence for Impact Investing.  This is not an easy process, nor is it an exact science.  It is undoubtedly easier in some cases than others; for example, measuring whether you are using less energy is far more straightforward than assessing whether an apprenticeship scheme is effective at reducing re-offending, due to the myriad of contributing factors into why someone is offending in the first place.

Our experiences of managing Nesta Impact Investments have re-iterated the importance of aligning commercial and social impact. In the pipeline of promising opportunities that we are investigating, the most compelling tend to be where there is complete alignment of the commercial and social goals. For example, take the health sector where technologies now allow the remote monitoring of patients with long term medical conditions which in certain cases have been shown to improve patient’s quality of life, or in the digital education sector where there are a growing number of on-line learning platforms that purport to raise attainment levels.  In either case the commercial and social goals are closely entwined.  A clear verified assessment of the offering’s ability to deliver on the stated impact goals is bound to improve the commercial proposition of the companies in question – essentially it’s the outcomes that the customer is buying.

As we reflect upon our initial learnings of running Nesta Impact Investments, it is clear to us that there is undoubted commercial value to accurate evidencing and measuring of impact.  It can lead directly to improved outcomes in sales, product development, competitive positioning and even raising investment  – all of which directly increase the value of a venture.

---

By Alex Hook – Nesta Impact Investments

This blog post introduces Nesta’s Standards of Evidence, a developed to test whether the products and services Nesta Impact Investments fund make a positive difference. This new approach aims to bring impact measurement in line with academically recognised levels of rigour, whilst managing to ensure measurement is appropriate to the growth trajectory of the products and services we invest in.

This week we launched Nesta Impact Investments. This new fund will support social ventures that can help address major social challenges. To ensure what we fund does make a positive difference we have developed a new approach: Standards of Evidence for Impact Investing.

For the fund good intentions are not good enough. We know that good intentions don’t lead to good outcomes. Programmes like Scared Straight or DARE, which have been found to be harmful to the young people they set out to serve, are good examples of this.

Yet we recognise that the demand for evidence needs to balance against ensuring innovation can flourish, that the bar is not set so far that it becomes an unwieldy milestone or a hindrance to growth and development. This is why we have developed Standards of Evidence which are tiered and incremental, helping grow the evidence behind products and services at a rate that is appropriate to them.

The diagram below outlines our Standards of Evidence for Impact Investing:

 

The Standards of Evidence are on a 1 to 5 scale. The starting point for all those we will fund is Level 1, this involves a clear articulation of why a product or service could have a positive impact. As the levels are progressed it will be expected that data is collected to isolate the impact to the product or service, the findings are validated externally, and by level 5, demonstrable evidence that the product or service can be delivered at multiple locations and still deliver a strong, positive benefit.

We have just published a report which outlines the Standards of Evidence in greater depth, but to summarise the key features are:

• The Standards of Evidence recognise the need to ensure the demand or evidence is appropriate for different stages of product or service development so that it doesn’t hamper innovation
• We know what we expect to see at each stage, but are not prescriptive about how individual organisations meet this, enabling them to select approaches appropriate to them.
• The evidence requirements are realistic and proportionate
• The different levels are intended to be dynamic and developmental

It is worth noting that standards of evidence are not new. There are numerous examples used in academia, such as the Maryland Scientific Methods Scale, or the Project Oracle Standards of Evidence used in youth services, which were the starting point for our own Standards.  Yet, as far as we can tell, this is the first time standards of evidence have been used in an investment fund context.

This means our Standards of Evidence for Impact Investing are something of an experiment, a new innovation in the field to assess the impact performance of investments. We therefore welcome feedback to help us reflect on the Standards of Evidence, recognising that we may need to adapt and change over time, all helping contribute towards ensuring that what we invest in has the biggest impact possible.

If you would like further details about the Standards of Evidence for Impact Investing please see our latest paper.

Further information about Nesta Impact Investments.