Measuring Your Social Impact: What the Research Says

These missions and goals are well-intended, but we haven’t yet come up with a good way to measure their impact. While research and practice have conceptually grounded social performance in theories of morality and social responsibility, measuring the social impact of a business entity requires a new and different approach. From our review of 71 academic research articles about different ways to measure social impact, we’ve been able to glean some recommendations that can help companies measure their impact more effectively and better communicate it to internal and external stakeholders.

Observations

To begin, we define social impact as beneficial outcomes resulting from prosocial behavior that are enjoyed by the intended targets of that behavior and/or by the broader community of individuals, organizations, and/or environments. This definition encompasses most current approaches to measuring social impact and recognizes that social impact involves many different phenomena and target populations, both now and in the years to come. Whether impact is generated by public, private, non-profit, or for-profit organizations, its measurement is not a function of  intentions, goals, identity, values, and missions (however noble they may be), but rather the actual outcomes resulting from their implementation,.

We explored these issues in a recent research article published in Entrepreneurship Theory & Practice. [1] We performed an electronic search to identify academic articles focused on social impact that were published in the top 50 business journals[2] from 1996 to 2016. We included a wide variety of relevant keywords in our search, including “social value,” “social impact,” “social return,” “environmental performance,” “impact measurement,” “triple bottom,” “social performance,” “non-financial performance,” “environmental impact,” and “social accounting.” From an initial sample of 273 articles, we settled on the 71 that were most relevant to our review.  Below are the main observations and practical takeaways from our research that may be useful to business owners seeking to generate social impact of their own. We also summarize our recommendations in the Appendix: Choosing Among Impact Measures (downloadable above).

Scope

The overwhelming majority of academic articles in our sample view social impact as a concept that can be generalized to many sectors, rather than as a highly specific concept that can (or at least should) only be compared between similar organizations. This multi-sector approach allows for developing standards that can be measured across industries. For example, ASSET4 ESG data provides environmental, social and governance (ESG) performance indicators on more than 6,500 public companies throughout the world. Similarly, MSCI (formerly KLD) scores describe the ESG performance of 8,500 companies worldwide. These types of databases enable investors to identify the degree to which companies are actively pursuing socially responsible initiatives, or whether they are avoiding them or being irresponsible[3]. While these databases provide standards for organizations to understand and report their impacts on the economy, environment and society, these standards often require companies to use the same metrics, which often is neither practical nor feasible. This leads to companies trying to shoehorn their internal measurement data into the categories required by the standardizing agency, resulting in confusion and misalignment. In short, the broader the scope of the industries, the less valid multi-sector comparisons will be.

The single-sector approach lets researchers study the social impact of one sector more deeply. This makes it far easier for them to examine the assumptions underlying the processes that result in social impact in that sector. For instance, this approach inherently assumes that the factors most important to social impact creation and destruction in the fast food industry are distinct from those in other industries.[4] The single-sector approach focuses in greater detail on the types of social impact that are most relevant in that particular sector, rather than seeking to address all types of social impact. Thus, while a single-sector approach can shed light on the factors and challenges that shape a particular industry or marketplace, its insights can’t be generalized to other sectors.

Process

A “theory of change” describes the process by which specific organizational activities lead to beneficial outcomes for society and/or the environment. While both the inputs and outputs are critical validating theories of change, we found that most researchers in our review measured either activities or impact, but not both. Similarly, business owners often theorize that if they initiate or change a certain business activity, it will have a beneficial effect on the community, the environment, or a certain social problem.  The problem, however, is that measuring an organization’s activities and outputs is easy and inexpensive, but measuring outcomes is not. We recognize that while organizations should always be able to envision specific outcomes from their activities and to measure them, sometimes it’s not feasible. Therefore, we propose that as a starting point, organizations should clearly identify (at least at the conceptual level) both the activities that they engage in and the outcomes that they believe will result from these activities. We also encourage organizations to connect the measurements that they make to the activities and outcomes that they have conceptualized. Then they can engage in a clear-eyed explanation about the discrepancies between their theory of change and their measurement.

Third-party Certification vs. Scientific Methods

Third-party standards and certifications (such as Fair Trade, ISO 9001, Rain Forest Alliance and others) assess the relative social impact of organizations and products and deem them “good” or “bad.” They do so by aggregating different types of social impact to form a composite metric that categorizes the  organization or product as “responsible,” “fair,” “social,” “green,” etc. and then certifies only the organizations that reach the benchmark. While this process can yield a fairly comprehensive assessment of a firm’s social impact, the problem is that even firms earning the same certification often vary widely in how (and how well) they achieve the intended impact, so it’s difficult to draw accurate comparisons. Also, firms that failed to get the certification can’t learn from firms that achieved it. The reasonable assumption underlying categorizations is that positively categorized organizations and products will have greater social impact than negatively categorized ones; however, their ease of use comes at the expense of more direct measures of social impact.

Social impact measures for quantification purposes are more scientific and rigorous. Taking a cue from the gold standard in social science – the randomized controlled trial (RCT) – this research compares the impact an organization has before and after implementing a specific action. This builds on the concept of “additionality” (i.e., the outcome would not have occurred without a particular intervention) that undergirds the RCT. This RCT-inspired approach requires specifying a clear counterfactual, do-nothing alternative, to which social outcomes related to a product or organization’s actions are compared, as well as clearly specifying measurement in terms of changes (e.g., improvements in conditions of interest) during a particular time period following a given activity. Proper use of quantification in social impact measurement is complicated and can be costly[5], but we believe is a useful ideal.

Many articles in our sample also aggregate multiple types of impact (i.e., social, environmental, political, etc.) into a simplified metric, regardless of whether they categorized or quantified their measure(s) of impact. Other articles keep the various types of impact separate, under the assumption that different types of social impact are not actually comparable (e.g., how many tons of CO2 are equal to a job created for the poor). Aggregation definitely has an appeal; it reduces multiple measures to a smaller number of measures that are more easily reported and compared. However, if improperly used, it can often oversimplify complex issues or even be misleading. So organizations should use caution when combining or aggregating multiple measures of impact.

Recommendations

Based on our in-depth analysis of these articles, we offer the following recommendations, which are meant to help both academics and business owners think more critically about social impact and help improve how it is measured in the future.

Articulate the Theory of Change

You should clearly state your assumed theory of change – the social benefits that you feel will result from a specific action.  Measurement can help to determine if the assumptions underlying your theory of change are accurate. More specifically, impact researchers studying activities should specify the types of outcomes that ought to be associated with these activities, the nature of the implied causal link, and their justification for not measuring those outcomes more directly. By specifying the link between activities and outcomes, we can gain clarity on the causal mechanisms. Business owners should also acknowledge both the positive outcomes and the reduction in negative outcomes of various socially focused activities and consider both in their measurement models.

Consider the timetable for impact

When a business clearly explains what it plans to do and estimates how long it will take for its activities to lead to desired outcomes, stakeholders can more readily understand (and accept) its strategy and timeline. Although results may not be measurable for a longer time than stakeholders may want, and it may be easier in the meantime to measure activities rather than their effect, it is important to be explicit in expectations for a few reasons.

For starters, the lag time between an activity and outcomes is likely to differ between activities. Consider, for example, that while emission reductions can be measured immediately following an activity, the impact of other activities, particularly those related to human development, might only be measurable over a person’s lifetime. Thus, we encourage practitioners interested in measuring social outcomes to specify both for themselves and for external stakeholders the time interval in which related outcomes will happen, as well as the rationale behind the timetable for measurement (e.g. “We anticipate that while the household income of workers will increase immediately, local educational impacts will not be observable for nine months due to school testing schedules.”)

Because it’s so costly, measuring some outcomes may not be feasible, requiring proxies to be used. In such instances, we advocate that businesses clearly specify both the ideal (but perhaps unmeasurable) outcomes as well as activities and outcomes that can be measured in their place. For example, an organization may state that while bringing employees out of poverty is one organizational goal, this is proxied by the percentage of employees' children that graduate high school, the percentage of employees who made contributions to a retirement savings account in the last year, and the percentage of employees who have completed a personal finance course. Another strategy is to employ long-term, qualitative methods to measure the full impact of an activity that cannot be measured quantitatively.

Identify the levels at which impact occurs

Practitioners should also identify the level(s) at which impact is expected to occur, as well as the level at which measurement actually occurs. These levels may include: individual, project, organization, community. If these boundaries are drawn too narrowly, researchers may fail to fully observe the outcomes resulting from organizational activities. This is particularly problematic if the outcomes include some negative results that are unforeseen and unmeasured. If the boundaries are drawn too expansively (e.g. measuring local poverty rates as a measure of a small organization’s economic impact), measurement may capture positive social impact beyond what was caused through organizational action.

Some (if not most) activities may not just have direct effects on the beneficiaries targeted by the activities, but also secondary effects at other levels of analysis as well. These may include benefits for non-users, legislative changes, etc., which invariably take more time to transpire. In such cases, businesses can follow the example of a noteworthy research study[6] that measures changes in income of coffee producers as well as the changes in outgroup prejudice and social trust that result from these changes in income.   

Note that social impact can be defined and measured at the product/project level. For example, lifecycle analysis (LCA) can measure the social impact of a product beyond manufacturing, including its use and disposal – forms of impact that firms often overlook.  Businesses should consider other players in the value chain when they think about potential impact and try to manage it, so that they can work with these partners to maximize impact.

Build better data through collaborative efforts

There has been some progress in social impact measurement. For example, B-Lab’s B Impact Assessment calculates actions and outcomes related to social responsibility across five dimensions (environment, community, workers, customers and governance) in order to create an overall “B-score,” which can allow firms to be compared with one another. While this multi-sector approach has gained currency in many countries throughout the world, it is most useful for firms that are trying to be viewed as exceptional in corporate social responsibility, and may not provide a good fit with the theory of change of many organizations focused on a specific social impact.

Because organizations focus on many types of social impact, we believe that similar organizations focused on similar social problems should partner with one another and adopt single-sector measures. Impact investing organizations have already begun work on creating measures suited for specific types of social problems. This work has the potential to organize what is already known by creating a repository where organizations share information related to the type of social impact they want to achieve: a written theory of change used for past interventions, specific social impact measures used, data collected, white papers based on lessons learned, questions, and concerns from each specific intervention. Relatedly, similar organizations can also work together to develop and adopt universal standards for the impact metrics now used by third parties based in different countries. For example, several sustainability-related tourism standards exist across different countries, which limits the potential for learning within the industry. By collaborating and agreeing on standards, organizations can learn from one another about what works and what doesn’t.

We also advocate for social sector organizations to work together with academics on a shared mission to improve the practice of social impact measurement. This type of collaboration is already happening in some cases. As one example, B Lab (creator of the B Impact Assessment and B Corp certification) has made B Impact data from certified B Corps available through Data.world and has shared even more detailed data collected with select members of the B-Impact Academics group of academic researchers. In addition, firms seeking to create social impact may want to appoint academics to their advisory boards. In such a capacity, they can direct practitioners to established measures or provide useful feedback. Even better, if an organization or group of organizations is willing to test the efficacy of different approaches to social problems and is interested in publishing the results, academics can bring research skills to the process (e.g., survey design, theoretical understanding, analytical modeling for complex interactions, data analysis) that will improve the evaluation process. This type of collaboration can be difficult to achieve, but the potential gains are significant.

We also summarize these recommendations in the Appendix: Choosing Among Impact Measures (downloadable above).