Dr. Kathryn E. Newcomer, Director, The Trachtenberg School of Public Policy and Public Administration

Jerry Wei provides great food for thought in his article on metrics. He is spot on in pointing that the search for measures to guide decision-making in the public and nonprofit sectors seems to be ubiquitous, overly ambitious, and conducive to “Metric Madness.” My over two decades of research on implementation of the Government Performance and Results Act of 1993 (the GPRA Modernization Act of 2010), plus my experience with helping many public and nonprofit organizations improve their measurement systems over this time same time, leads me to simply add a few more points to reinforce Jerry’s prognosis.

First, perhaps the greatest challenge to developing useful measures and measurement systems is ensuring adequate and candid communications among the relevant stakeholders about both why measures are needed, how they will be used, and what to measure. Developing measures and appropriate measurement approaches takes time – time to get the relevant stakeholders talking candidly about the drivers, measurement tools, timing and level of transparency of measures, and to develop and improve reliable collection procedures.

Second, there is a delicate balancing act in terms of using performance measures for accountability versus learning in order to improve programs and policies. As Jerry points out, in many (most!) of the public sector examples, measures are used to hold managers and service providers accountable. Yet evaluators (such as me!) and proponents of data-driven decision-making and data analytics urge that managers use the data collected to support learning in order to improve public and nonprofit services and outcomes. Given the capacity available, reporting the measures “up” to funders typically trumps allocating time to internal learning. Most nonprofits secure funding from multiple sources, thus the multiple and distinct reporting requirements can be demanding and overwhelming.

Finally, commitment to learning from data collected about program operations and outcomes for those served by public and nonprofit programs is sorely needed. Agency leaders, budget office staff, program managers, boards overseeing program and agencies, and front-line service providers need to embrace and reinforce the value to be reaped from learning from analyses of the “metrics” that are collected.

Developing and sustaining commitment to getting value form metrics is challenging, and shaped by the organizational cultures within the public and nonprofit agencies. Learning-focused cultures are needed in which:

• There are tangible and/or intangible incentives to use data to inform decisions
• There is deeply held commitment to achieving mission-driven goals throughout the organization
• The theory of change underlying programs and policies is understood
• Errors are treated as opportunities for learning, and not simply punished
• Attention is devoted to both inward and outward-focused learning – that is, there is also ongoing learning about the contextual factors affecting the achievement of desired program outcomes outside of the control of the program.

Moving toward shaping and sustaining learning cultures is our next frontier!