in Blog

Avoiding Analysis as Usual

High-performing organizations are re-awakening to an innate truth: analysis is not an end unto itself. Analysis is a means to advance the performance of the enterprise. Treating analysis like an enterprise asset to improve performance is not easy. Many governments know they need analytics, but they park the capability in siloed programs and invest sparingly in the practice. So how can governments build (and scale) enterprise-level analytics capabilities? To start, they can avoid these six behaviors which dilute government’s ability to harness the scalable power of analytics:

  1. Zooming in before zooming out
  2. Being too big to learn
  3. Drowning in Generalists
  4. Confusing “reporting” with “analysis”
  5. Outsourcing the hard stuff
  6. Ignoring technological progress

Zoomed In (Too Close)

The problem: Many governments have zoomed too closely into a particular program or issue and have taken too narrow a view of the value of their data collection and storage. They are busy building systems for short-term processing, auditing, and business needs. They are not paying enough attention to the strategic analytics and operational decision-making components of their data. This kind of tunnel vision can lead to systems that aren’t interoperable, don’t work on uniform standards, and often neglect to collect or maintain key pieces of information to facilitate analysis at a more global level. Changing course and zooming out doesn’t just take time and attention, it takes a reapplication of resources and emphasis on long-range strategic planning for IT and digital infrastructure expenditures.

One solution: The City of Boston rejected the false choice between zooming in and zooming out with the launch of City Score in 2016. As co-creator Daniel Koh explains in his TedX talk, the Boston team built an algorithm which aggregates the city’s performance metrics into one number – giving the city’s leadership a zoomed out view of performance and the ability to zoom in on the underlying causes of underperformance. Such composite measures are not without their faults, but City Score forces the city to see itself as a unified organization whose programs all contribute the city’s performance. Since it’s launch, the city cites improvements to EMS response times and a 21% improvement in street-sign installations.

Too Big to Learn

The problem: Governments are big. They employ more people than most companies and have more product lines than most people will ever see or care about. To run a government is to know how to pick up trash, educate children, house the homeless, maintain parks, pave streets, and respond to 911 calls. To get the work done, governments create multiple departments, programs, and subprograms. As government grows, it becomes filled with siloes, programs, and people walled off from one another to allegedly maximize the efficiency of each without regard to the whole. These siloes often create much larger inefficiencies and literally separate the people with key insights from applying those insights horizontally. Cross-agency transfers are rare. Even cross-department transfers can be unusual as people serve their whole city career in one or another department or agency. When all you know is sanitation, your understanding of the broad range of services and the entirety of the government mission can get lost, which can be a problem at a leadership level.

One Solution: The cities of Denver and San Francisco are proving no government is too big to learn with their creation and continuous improvement of city-wide training programs. Denver’s Peak Academy relentlessly trains municipal employees from every department to identify waste, eliminate it, and analyze data in a way that benefits the organization. In fact, Peak is rolling out an entirely new course in 2017 dedicated to helping all city and county employees make better use of city-owned business intelligence software.  San Francisco has a more data-intensive focus with their Data Academy, which helps city and county analysts explore, refine, and enhance their skills in data analysis and visualization.

Drowning in Generalists

The Problem: On the surface, it looks like governments are awash in analytics, because “analyst” is one of the most ubiquitous and amorphous position titles in government. In a typical government organization, many people hold a title which includes the word “analyst,” but too few are actually charged with conducting quantitative analytics on the programs they support. In most cases, analyst is shorthand for someone who takes on anything. They can manage projects, write policy memos, develop budgets, manage contracts, and plan events. You name it, they can probably do it. The one thing they cannot commonly do: advanced quantitative analytics. It has simply been too long since they learned to do regression or sampling and the organization usually contracts out the tough analysis to a private firm. Even if they do have the right skills, analysts are often assigned to a silo within the organization, lack access to data from multiple programs, and are rarely rewarded for innovative analysis. These constraints limit their ability to apply insights horizontally or find connections between programs.

One Solution: With foundational support from Bloomberg Philanthropies, the City of Los Angeles, CA created an innovation team in 2015, and the team is not just filled with typical public policy folks. A quick scan of their composition highlights diverse backgrounds in data science, product development, IT consulting, design, visual communications, and large-scale project management. Together, they use “a relentless focus on partnership, transparency, data, users, and outcomes” to provide in-house consulting to city leaders on some of LA’s toughest problems: the displacement of businesses and residents and changing neighborhoods.

Reporting ≠ Analysis

The Problem: In too many organizations, meaningful analysis has been replaced with generating routine performance reports. There are even “reports specialists” whose function is to continuously regenerate the same dashboard or scorecard that management has come to expect. But performance reporting is not the same as analysis and confusing the two does injustice to both. For example, a routine manager’s report showing a consistent number of daily 311 calls can overlook important trends in call volume, like fluctuations in the time of day people are calling. Without analyzing the whole picture, performance reports tend to oversimplify program dynamics and give a false sense of security to busy managers.

One Solution: In his book, Creating a Data Driven Organization, Carl Anderson emphasizes the Analytics Value Chain, where people solve problems by using data to generate performance reports and alerts, which trigger analysis that results in action. Kansas City, MO (KCMO) is a city that takes that analytics value chain very seriously. For years, KCMO has studied the progress maintaining thriving neighborhoods and healthy communities in its KCStat progam, but the city is not satisfied with its performance on property violations and enforcement. So, they are using their performance reports to drive deeper analysis in partnership with the Center for Government Excellence (GovEx) at Johns Hopkins University. Together, GovEx and KCMO are finding a predictive model for property violations which will help KCMO better target its resources toward high-impact interventions.

Outsourcing the Hard Stuff

The Problem: The average analyst has a graduate degree and learned to do statistics and program evaluation before joining the ranks of government. But for some reason, government started outsourcing advanced analytics to private sector firms, leaving the municipal staff to manage contracts instead of practicing analysis or learning new skills. When governments outsource the analytical capabilities without creating a strong partnership, they do themselves a disservice in the long run. As technology becomes cheaper and data becomes more widely available, having the partnerships and capabilities on hand to deliver insights is critical to long term success. The government’s people, partnerships and data should be treated like strategic assets. Rewarding analytical talent by promoting an analyst into a mere contract management position is an outdated model for professional growth and development.

One Solution: The City of New Orleans, LA is setting the standard for finding innovative partnerships with the private sector and turning those partnerships into city-owned assets. In a 2016 interview with Gov Innovator, NOLA’s Director of Performance and Accountability described his seven-person staff as “use case truffle pigs” who find the right opportunities and leverage ongoing pro-bono partnerships with corporations, academics, and nonprofit organizations to develop sustainable and repeatable solutions for the city. Such partnerships have helped improve smoke detection programs, code enforcement, and health care for residents.

Ignoring Technological Progress

The Problem: Data collection, storage, integration, and analysis used to be much harder than they are today. In today’s world, technological advancements streak by with the blink of an eye. Look away for too long and important data technologies and practices will pass right by. Too many governments are ignoring important advances in technology, which allow for large scale analysis of data from various sources in ways previously impossible or impractical.

One Solution: The City of Long Beach, CA is one of many cities making the transition from data-publisher to data-storyteller through a greater utilization of new features within its existing technology platform. After years of publishing datasets on an open data portal, the city is launching DataLB: a public GeoSpatial & Open Data Portal for exploring, visualizing and downloading data. According to the city’s Mayor, this shift will leverage technological advancements to help residents understand the data, not just access it.

Conclusion

As your organization uses analysis to improve performance, remember that analysis is simply the pursuit of understanding. So avoid any practices that prevent you or your organization from fully understanding the challenges you face. In my experience, zooming in before zooming out, being too big to learn, drowning in generalists, confusing “reporting” with “analysis,” outsourcing the hard stuff, and ignoring advancements in data technology are all practices that hinder long-term performance. Avoiding these pitfalls can help you harness the scalable power of analytics for your organization.