Making Shared Measurement Work

I have been excited to hear about many of the collective impact initiatives going on across the country and around the world as a way to address complex social problems. As described in an FSG article in SSIR, a collective impact approach is one where a group of multi-sector stakeholders come together to solve a complex social problem, with an agreed-upon goal, common metrics, and differentiated and mutually reinforcing activities. A cornerstone of such initiatives is shared measurement.

The aim of shared measurement is to focus different stakeholders’ attention on a set of common measures as a way to bring alignment, focus, and increased impact to challenging social problems. For example, increasing high school graduation is a commonly agreed-upon indicator of improved educational outcomes. I have seen the process of developing shared measures play out in the context of the entire education continuum, as part of a “cradle-to-college” initiative in the Seattle region.

As detailed in an earlier blog post by my colleague Fay Hanleybrown, identifying shared measures is an important part of a complex and multi-part process. The steps that follow defining common measures are as important as identifying the measures themselves, as these steps ensure that the data is reported, accessible and useful, which is the key to supporting data-driven, large-scale change. These steps include:

>Assess if, where, and how the data is collected: Once shared metrics are identified, the first step is to understand if and how the data is collected. It is important to go through the process of mapping out the data source, frequency of collection and reporting, how the data is disaggregated (e.g. by race/ethnicity, income), whether it is publicly available, or if obtaining the data would require a data sharing agreement. For example, in Washington State, data on high school graduation is publicly available from the Office of Superintendent of Public Instruction, is reported by district and school, and is disaggregated by race, income, and other student sub-groups.

>Determine how the data will be used: Once the source for each metric has been identified, it is important to consider how the data will be used. For example, with high school graduation, data could be used to track the progress toward a specific goal or target–such as increasing the on-time graduation rate to 90 percent. Collecting annual graduation data over time for different schools in a district would show which schools have met or are approaching the goal, and where progress has not been made or ground has been lost. Data can also be used to inform practice. This would mean looking at high school graduation data over time to identify those schools that have seen progress in increasing their graduation rate to explore “under the hood” to understand how and why this progress has been made. Ultimately, the practices of those schools that have improved could be shared with schools that have seen less progress.

>Where needed, build relationships to collect data: For those data that aren’t readily available through public data sources, relationships are the key. Data sharing agreements may be required to get access to the needed data, which requires support from various stakeholders. Developing trust and understanding of how data will be used are critical for getting access to data and creating buy-in for the work. For example, a college mentoring programme could pursue a data-sharing agreement with a school or district to understand how the students they serve are doing with respect to graduation requirements, and if their students are on-track to graduate. Access to this level of data–which is more specific than whole-district or school-level data–often requires the execution of a data-sharing agreement to govern how data can be shared, stored, and used.

>Make data accessible and relevant: Once the data is collected, it is important to make sure it is useful, relevant, and digestible to the target audience(s). To track progress of a collective impact initiative, for example, it is useful to display data relative to a time-bound goal or target. Displaying a district’s graduation rates of 74 percent in 2010 and 78 percent in 2011 alongside a goal rate of 90 percent in 2018 demonstrates how that district is performing with respect to a specific target, and shows how much progress is still needed. To improve practice, providing more granular data is necessary to inform decision-making. For a college mentoring programme, for example, it would be useful to provide reports on which students are [and aren't] on-track. This would provide specific information about which students need what type of support to be able to graduate, and could help the programme to better target its attention and support.

Identifying shared measures is an important part of getting participants in collective impact initiatives on the same page and align them in their thinking about what they aim to collectively accomplish. But it’s only the beginning!

With thoughtful consideration of each of these steps in the process, data on these metrics can ultimately be collected, analysed and used to drive further progress.

Where have you seen shared measures effectively adopted and used to solve complex problems?

By Nathalie Jones, FSG Consultant (Nathalie has worked with foundations and nonprofit organizations on a number of evaluation, strategy, and collective impact engagements. She is based in FSG's Seattle office.)

The thoughts and opinions shared here are of the author.