Pereiti į pagrindinį turinį

Guidance note: Managing Measuring and Reporting Impact

Overview: what are some key considerations for social impact measurement and reporting?

This policy guidance note describes different considerations when it comes to social impact measurement by social enterprises and other stakeholders. It is structured around good practice statements included in the action area “Managing, Measuring and Reporting Impact” in the Social Entrepreneurship component of the Better Entrepreneurship Policy Tool developed by the OECD Centre for Entrepreneurship, SMEs, Regions and Cities and the Directorate-General for Employment, Social Affairs and Inclusion of the European Commission: www.betterentrepreneurship.eu

The key “raison d’être” of social enterprises is to pursue a social or societal goal and, by doing so, generate social impact, whether for society at large, the environment, or specific target groups. Today’s societal challenges – from climate change to growing inequalities – call for new ideas that can generate both economic growth and social value. Social innovation and social enterprises can play an important role in this regard.

In light of the above, common responses worldwide have been to launch support programmes like social innovation subsidies, funding instruments, and/or educational programmes to encourage successful social entrepreneurial outcomes. Such initiatives unescapably lead to questions around how social impact will be defined, managed and measured. For example, which social entrepreneurs will be selected for awards by a support agency like UnLtd UK Who will a public innovation agency like Flanders’ Agency for Innovation and Entrepreneurship provide its social innovation subsidies to? Who will an impact investment fund like SI2 invest in? Which candidates will be admitted to Ashoka’s Impact Programme? In each case, a systematic and coherent approach to evaluating the potential, actual and/or future impact is essential.

While there are several measures that provide a quick feedback on financial viability (albeit not without their own underlying complexities), like cash planning, the surplus or profits generated as a share of revenues, or company growth, when it comes to measuring and assessing social impact, things get a lot messier. Social enterprises’ missions are rarely single-headed, but instead multifaceted; they are rarely static, but evolving. Moreover, true impact is commonly observed over time rather than immediately.

There is, as of today, no broad-based consensus on the best measurement and reporting techniques and metrics to use by social enterprises or other actors in the ecosystem. While Social Returns on Investment (SROI) and Social Audit Network Approach are two well-known examples that have received widespread publicity, the adoption rate of these standardized approaches has been strikingly slow so far1. Nevertheless, there seem to be some broader consensus regarding the use of a common process to define one’s impact measurement and reporting approach. The basic principles and guidelines for such a process are clearly summarized in the report by the European Commission's Expert Group on Social Entrepreneurship (GECES): "Proposed Approaches to Social Impact Measurement", published in 2014, which is further in line with practical advice provided by the European Venture Philanthropy Association.

Given an increasing interest in social impact measurement, the techniques available have multiplied over the past two decades, stimulated by a growing number of consultancies, funds, crowdfunding platforms, think tanks, etc. The growing impact measurement scene has also been shaped by other influential trends, like that towards greater transparency and accountability (especially following the 2007-2008 financial crisis), which has led the way for instance to certifying agencies like B-Lab to help businesses (so-called B Corps) to audit and certify their social and environmental performance. Another influential trend has been the rapid advances in ICT. Thanks to new technologies allowing for low-cost ways to collect information directly from end-users of products and services, like for example GPS mapping tools and mobile phone surveys, social enterprises are able to measure their impact in new, bottom-up and more inclusive ways.

Nevertheless, impact measurement and reporting has also drawn its fair share of critics. Some have long worried that impact measurement steers attention away from hard-to-measure social performance indicators in favour of those relatively easier to measure. Others warn that the “straightjacket” of an impact measurement framework can render organisations less agile and unable to adapt to unpredictable twists and turns. Others still have held that measurement, monitoring and reporting efforts often require investment disproportionate to the actual impact achieved, particularly when initiatives are still at an early-stage (referred to as the “proportionality challenge”). Finally, there is a growing unease about the mushrooming of impact measurement consultants whose expertise is difficult to gauge and whose services can be prohibitively expensive for social enterprises.

Policy-makers are encouraged to be mindful of the various concerns that critics have raised. At the same time, their focus should remain on the end result they aim to achieve by actively promoting measurement and reporting, without imposing an unnecessarily heavy burden on social enterprises. By diffusing successful practices and embracing the innovations that technological advances and societal trends bring, policy-makers can help propel the development of better and more cost-effective measurement and reporting practices.

Policy levers for managing, measuring and reporting impact:

Policy Levers:

  • Help fast-track the diffusion of successful practices, valuable insights and knowledge about impact measurement and reporting.
  • Encourage a shared impact measurement system and continuous communication among relevant stakeholders.
  • Mainstream and implement high-quality impact assessment approaches, within public institutions.2

 

Pitfalls to avoid:

  • Actions that do not sufficiently incentivise people to engage. Calls for voluntary participation and/or contributions (like joining a learning platform) need to be behaviourally smart.
  • Seeing improved measurement and reporting as an end in itself rather than a means to an end - stakeholders should act on the insights made visible by impact measurement and reporting, not just publish them.
  • Waiting until a perfect solution has come. Instead, get started and move ahead, while learning by doing and from others. Thinking and practicing impact measurement and reporting should never stand still - it should always be evolving.

 


1 When it comes to social performance measurement, it seems that most social enterprises regularly use a simple count metric, such as the number of beneficiaries served, the number of people helped find a job, etc.

2 For two useful reference documents, see Social Experimentation: A methodological guide for policy makers, a report commissioned by the European Commission in 2011; and Policy Brief on Social Impact Measurement for Social Enterprises, prepared by the OECD in 2015.

Guidance per assessment statement

7.1. Social enterprises have access to methods for measuring and/or reporting impact.

We invite you to consider the various means available to social enterprises and other stakeholders to acquire information about impact measurement and reporting, and to act on such information. Have there been efforts to collate relevant information on available measurement and reporting techniques online or offline? Which additional measures are currently underway to effectively help social enterprises act on this information?

 

Why is it important?

It is essential that social enterprises and other interested stakeholders have access to good information regarding the methods and approaches that exist for measuring and/or reporting impact. To facilitate social impact measurement and reporting, public authorities could consider providing freely accessible online resources with existing social impact indicators and metrics that social enterprises can use.

Yet, there is no guarantee that social enterprises and stakeholders will (or are able to) act on such information. Different barriers, like whether information is available in local languages or whether social enterprises have enough resources and capacity to pursue this endeavour, will have a significant influence on the take-up and actual use of information and available resources. An extra push or incentive may be needed, for example through grants to help cover the costs of social impact measurement and reporting, including for its further development.

In order to score high, in your context:

  • An open-access online repository of methods for measuring and/or reporting impact is available in the native language.
  • Affordable support is available.
7.2. The impact metrics are co-constructed with the social enterprise community.

We invite you to consider the impact measurement and reporting techniques widely used in your context, and ask yourself how they were developed. Were they developed in a co-creative manner (i.e. through workshops or open debates including diverse stakeholders)? Or were they copied from existing blueprints without contextualization? Were they put together by a limited group of stakeholders, be it top-down public sector representatives or academics, or more bottom-up by social entrepreneurs themselves?

 

Why is it important?

Measurement and reporting techniques that emerge from a co-creation process are more likely to incorporate and accommodate diverse interests and viewpoints, and are thus more likely to be used by relevant stakeholders over time. The co-creation process itself is also likely to help ensure that the techniques are being adopted, tested and refined by diverse stakeholders. In contrast, measurement and reporting techniques that were developed by only one type of stakeholder are more susceptible to criticisms. Certain groups may feel that the techniques do not reflect their own needs and realities. As a result, over time these techniques are less likely to be widely adopted and used.

From an ecosystem point of view, the greater the alignment in approaches to impact measurement and reporting across different intermediary organisations that support social enterprises through their lifecycle (e.g. funds or capacity support organizations), the easier it will be for social enterprises to navigate the sea of different approaches . For instance, if the social enterprise needs to change its approach to impact measurement depending on which actor it speaks to, then this is costly and time-consuming.

From an internal perspective, one of the most valuable benefits that social enterprises and other actors like funds, network organisations, or public sector representatives can gain from co-designing an impact measurement and reporting approach is arguably the process itself. In the co-design process, they are pushed to clarify their mission and vision (e.g. What is their purpose? Where are they headed?) and their theory of change (how will they get there?). Some embrace this process to create a shared language (vocabulary) together with their staff and other stakeholders, pinning down their areas of priority, their respective roles, and the types of change they aspire to achieve in the short, medium and long run. Whilst this process can be quite intense and time-consuming, experiences show that the more inclusive and creative an organization allows this process to be, the stronger its end-result: an approach to measurement and reporting that tightly fits with the values of relevant stakeholders and the values of the organisation (organisational culture)3

Pointers to help tick the appropriate score

In order to score high, in your context:

  • Social enterprises and their coordinating bodies are closely involved in defining impact measurement and reporting techniques.
  • There is a platform for dialogue among all relevant stakeholders.

 

Good Practice Examples

Strive Partnership (USA)

StrivePartnership, part of the national network StriveTogether, started as one of the first cradle-to-career community in 2006 in urban school districts in Cincinnati, Ohio, USA. By bringing together local leaders to improve education in the region’s urban core, the partnership sought to increase student success in three public school districts. More than 300 cross-sector representatives joined in the effort, including school district superintendents, business and non-profit leaders, city officials and university presidents.

StrivePartnership didn’t create a new program or raise more money. Instead, to coordinate practices and direct resources towards solutions that work for kids, they used “continuous improvement” - a methodology often used in engineering and health care. Its stakeholders agreed on a common set of goals, outcomes and indicators, and then analysed and shared data to track progress.

As proof of the partnership’s collective impact, 10 years after its creation, the StrivePartnership became the first in the StriveTogether network to improve on over 85 percent of key indicators on student success.

Calgary Homelessness (Canada)

Calgary Homelessness is another collective impact platform launched in Calgary 2008 by a multi-stakeholder leadership group who knew they could end homelessness if they tackled it strategically. Their strategy was to shorten the stay of those using emergency service through a “Housing First” model. Housing First quickly moved people into appropriate housing, where they received support and help to address the issues that led to their homelessness. The Calgary Homeless Foundation (CHF) act as the backbone organization overseeing implementation of the strategy along with agencies in the homeless service sector, businesses, governments, the faith community, donors and Calgary citizens. CHF also promotes a so-called “System of Care Approach”. To implement the approach, they use a system planning framework to ensure that programmes respond to the needs of diverse clients while using a common assessment tool to determine needs and track progress. The framework also implies having clear and appropriate performance measurement indicators at both program and system level. Finally, they promote the use of real-time data to monitor the performance of the system.

 

 

3To illustrate, it is quite common in this process that social enterprises realize that they have drifted away from their core mission, adding on more and more activities (often under the impulse of funding opportunities), without maintaining sufficient overview and coherence.

7.3. Awareness-raising initiatives on impact measurement and/or reporting impact exist.

We invite you to reflect on the level of awareness about the value of impact measurement and reporting and about available approaches in your constituency. To help you assess this dimension, ask yourself whether there are initiatives designed to help raise awareness about impact measurement and reporting among social enterprises and their stakeholders.

 

Why is it important?

Lack of awareness about the value of impact measurement and reporting among different groups in the social enterprise ecosystem can create tension and lead to misunderstandings. For instance, when an investment manager is more accustomed to dealing with mainstream entrepreneurs and not aware of the specificities of social enterprises, he or she may find it difficult to adequately evaluate the social impact measurement and reporting that a social enterprise carries out. As a result, it may be valued too lightly. In the reverse case, social enterprises may be under the impression that they have to strengthen their impact measurement and reporting practices simply to please investors, but do not see what is in it for them. To address this, awareness-raising campaigns must be tailored to different circumstances and target audiences to ensure that all relevant parties benefit to the greatest extent.

 

In order to score high, in your context:

  • Campaigns illustrating the benefits from measuring and/or reporting impact also consider the challenges and capacity of social enterprises.
  • Stakeholders who design impact measurement and/or reporting requirements are made aware of the specifities of social enterprises.
  • Social enterprises are made aware of the availability of resources and approaches to measure and/or report impact.

 

Good Practice Example

Trends and Dynamics in the Portuguese Eco-system for Social Entrepreneurship (Portugal)

In a recent multi-country survey, Portugal stood out for its high share of social enterprises that measure their social performance: 97% of social enterprises in Portugal appeared to do so, versus 65% overall (SEFORÏS, 2016). Like in all other countries surveyed (except for Sweden), Portuguese social enterprises favoured measuring social performance by counting the numbers of beneficiaries or clients served.

One key reason behind the widespread awareness and practice in relation to social performance measurement in Portugal is that social enterprises have to report their social impact to the public authorities funding their activities, since a large share them either receive government grants and/or sell their products or services to government (70% of the surveyed enterprises).

Other initiatives in Portugal also contribute to greater awareness of social impact measurement amongst stakeholders in general, and social impact investors in particular. To illustrate, the Portuguese Social Investment Task Force, was launched in July 2014 as a 12-month initiative funded by the European Commission, aiming to promote meaningful and informed discussion about the potential of social impact investment in Portugal while gathering leading actors across social, public and private sectors. The resulting Portugal Inovação Social represents a key milestone in the promotion of social investment in Portugal. Established by the Portuguese Government in 2015, and approved by the Cabinet Council, Portugal Inovação Social was endowed with EUR 150 million from the European Structural Funds under the scope of the Partnership Agreement Portugal 2020. As such, it has a strong mandate and substantial resources with which to drive the development of the social investment market in the coming next years.

For more information, please see Trends and Dynamics in the Portuguese Eco-system for Social Entrepreneurship (Portugal)

7.4. Impact measurement and/or reporting features are discussed in the public debate and feed into policy-making.

We invite you to consider to what extent impact measurement and reporting features in the public debate and feeds into policy-making. Is the evidence on impact measurement used to inform a critical discussion about the value added by social enterprises?

 

Why is it important?

Impact measurement and reporting should not be an end goal in itself. The insights generated should serve as a means to identify areas that require improvement or areas that show sufficiently promising results to be scaled. To promote this, policy-makers can commission high-quality impact evaluations of key policy initiatives supporting social enterprises. Importantly, sufficient resources must be foreseen to ensure that valuable insights can also elicit lively debate and ultimately stimulate appropriate action and evolution of the field. There is widespread concern that high quality impact assessments all too often fail to improve public debate and policy-making. Some potential reasons are:

1) the gap between scientific language and popular understanding, which is often left too wide;
2) that acting on findings often requires changing the status quo, which is notoriously difficult to do; and
3) that many academics do not feel sufficiently compelled or comfortable to engage in public debate.

In order to score high, in your context:

  • The public sector systematically evaluates the impact of its key policy interventions.
  • Evidence produced by impact measurement is used in public debates.
  • Research on impact measurement is promoted.

 

Good practice example

What Works Centres (UK)

In 2013, the UK government launched a network of independent What Works Centres, each focused on a key policy area receiving substantial public funding. The network is underpinned by the principle that good decision-making should be informed by the best available evidence. If evidence is not available, decision-makers should use high quality methods to find out what works. What Works constitutes one of the first times that the UK government has taken such a systematic national approach to prioritising the use of evidence in decision-making.

The centres help to ensure that thorough, high quality and independently assessed evidence shape decision-making at every level. They do so by:

  • collating existing evidence on how effective policy programmes and practices are;
  • producing synthetic reports and systematic reviews in areas where they do not currently exist;
  • assessing how effective policies and practices are against an agreed sets of outcomes;
  • sharing findings in an accessible way;
  • encouraging practitioners, commissioners and policy-makers to use these findings to inform their decisions.

Existing What Works Centres include: Centre for Ageing Better (improving the quality of life for older people), College of Policing What Works Centre for Crime Reduction (crime reduction), and What Works Centre for Local Economic Growth (local economic growth), The Education Endowment Foundation (EEF) (educational achievement), and the Early Intervention Foundation (early intervention).