If you’ve spent time on our website then you know that we really love our numbers. And we’ve seen a lot of data programs that clearly do as well! In our work with cities, there are many terrific examples of how offices are tracking metrics from their open data programs. The most common measures relate to the number of visitors to the open data portal, data downloads or data views created. There are also a number of quantitative ways to track the impact of an investment in analytics capacity or a performance focus. Denver’s Peak Performance program, for example, captures the value of their targeted process improvements in terms of the money or time they save. This regular practice allows them to roll up the total value of the improvements to city process resulting from their program.
While we clearly capture valuable information from quantitative measures like these, even the most complete set of numbers misses something important about what’s happening out there as a result of your data program. Fundamentally, if you know that your broader goal is to improve people’s lives, how do you measure that?
Numbers can’t paint the whole picture when it comes to qualities of human experience. Getting at the stories that demonstrate the effect your data program has on your residents or constituents is a very important part of your impact tracking. However, where do you start looking for the kinds of stories that could help you see how your data program is ultimately directly affecting your residents?
To help cities identify moments where they might have the start of a good story about the effects of their data program on residents, we developed the Effects on Residents survey. This survey asks you to consider four different ways that you might have used data that would have direct effects on people in your community.
The survey asks:
- Have you recently made a decision informed by data? That is, did you analyze data to help decide about where to put resources, political capital, or to change a program? Even in cities that don’t have a large analytic program this will still often happen in the context of budget decisions, although cities rarely use the opportunity of budget analysis to break out individual stories.
- Have you implemented a decision made in a performance meeting? If you have a Stat meeting, performance meeting, or regular strategy session where you review metrics and figure out improvements, then you have a lot of potential decisions to use as the basis for a story.
- Have you held a public event – such as a meeting or competition – that prominently featured your data? If you encouraged members of the community to sit down and look at your data with you in order to build trust, or you asked for the community to provide their ideas based on a city dataset, you used data to strengthen your bonds with your community.
- Have you provided data to external stakeholders in a way that benefited your residents? Maybe you used data to help other people make a decision that helped people in a different way: for example, you may have provided a funder with data to show the effects of one of your programs, or you might have provided data to university researchers who developed good recommendations for you.
This set of questions does not by itself create the stories you need to tell about the effects of your data program on your residents, but it does point to where you can begin to identify them. By using data in decisions, in public events, or in research partnerships that produce public benefit, your data program changes the way you serve your community. Telling the story of those ways that your data use has affected residents is just as important a way to demonstrate impact as any quantitative measure.