OMB Releases Plan to Elevate Performance Evaluation

The Office of Management and Budget (OMB) released a memo to federal agencies on Oct. 7 that outlines a new initiative to bring a renewed emphasis and additional resources for program evaluation within agencies. Although this initiative is not a comprehensive plan to reinvigorate performance measurement in the federal government, it will help correct many problems that kept previous performance systems from creating real improvement in government performance.

The memo is entitled "Increased Emphasis on Program Evaluations" and details a three-part plan to help agencies develop better systems to conduct "rigorous, independent program evaluations." The plan includes giving better access to agency program evaluations on the Internet that are both in progress and planned for the future, re-launching an interagency working group on evaluations, and a voluntary pilot program to provide additional resources to fund rigorous program evaluations or strengthen evaluation capacity within agencies. Each of these three policy changes will help to improve performance evaluations within agencies and encourage better use of performance information.

Posting more information about federal evaluations online: OMB will begin working this fall with federal agencies to expand access to information about program evaluations. The goal will be to "make researchers, policymakers, and the general public aware of studies planned or underway" that examine if a program is making the grade or evaluate the effectiveness of other approaches and strategies that achieve the desired outcomes.

This plan mirrors the Obama administration's general commitment to a more transparent government but also helps to decrease a tendency to protect agency prerogatives in performance evaluations. During the Bush administration's use of the Program Assessment Rating Tool (PART), there was reluctance among some agency staff to share all of their internal performance evaluation materials and results with OMB. In addition, OMB often dismissed evidence and evaluations that were shared as unsatisfactory. The requirement in this memo to simply make program evaluations public should help to create a more positive dialogue about performance evaluations and results.

What's more, this requirement has the potential to become as useful a repository as one that allows the public to search all government-funded clinical trials provided by the National Institutes of Health (see ClinicalTrials.gov [1]). While the memo does not explicitly state that OMB will develop a central repository for all government program evaluations, allowing for the capacity to search evaluations across agencies and departments that focus on a central goal or issue (alleviating homelessness, for instance) would allow for better coordination and communication between agencies that run programs with similar objections.

Interagency Evaluations Working Group: The second part of the memo states that OMB is going to re-constitute an interagency working group of program evaluation experts from the Domestic Policy Council, the National Economic Council, and the Council of Economic Advisers. This working group will help build agency evaluation capacity by developing a network of experts inside and out of the federal government, share best evaluation practices from across the federal government, develop techniques for using evaluation and performance data to continually drive improvement, and potentially develop government-wide guidance for program evaluation practices.

There are three key points made within the language of this section of the memo that are improvements over previous performance efforts. First, it is clear the Obama administration understands a one-size-fits-all approach will not succeed. The memo references that different evaluation methods and structures are necessary for different types of programs and objectives the government is trying to achieve and that agencies need flexibility within government-wide evaluation guidance to "adopt practices suited to their specific needs."

Second, there is an acknowledgment, albeit subtle, that many agencies do not have the resources or in-house expertise to develop "strong, independent evaluation offices." This acknowledgment is bolstered by the fact that OMB intends to make additional resources available to a limited number of programs to develop their own evaluation systems.

Finally, in both the first section of the memo and the second, OMB opens the door to potential collaboration between government and outside experts to design and implement robust program evaluation studies. This could potentially allow those outside the government who are responsible for implementing government programs, as well as experts from academia or other sectors, to work more closely in designing evaluations that are targeted and useful for those implementing federal programs.

Voluntary FY 2011 Evaluation Initiative: The third and final aspect of the memo is a voluntary program that invites interested agencies to submit additional information along with their FY 2011 budget materials to win more funding for high-priority evaluation activities. OMB plans to award funding to 20 "rigorous program evaluations across the Federal government or [to agencies] to strengthen agency evaluation capacity."

Agencies must submit additional information to OMB that includes an assessment of evidentiary support for budget priorities, new proposals for rigorous evaluations, an assessment of agency capacity to conduct rigorous independent evaluations, and the identification of statutory requirements that may have the unintended effect of keeping agencies from conducting rigorous evaluations and assembling evidence about what is working within agencies.

This third part could almost be called a pilot program since it is initially being implemented in a limited way. Only 20 awards will be made to agencies, and the scope of performance measurement activities this will support is restricted to "social, educational, economic, and similar programs" that support life outcomes of Americans. The memo specifically mentions that four very large areas – procurement, construction, taxation, and national defense – will not be considered except on a case-by-case basis. OMB Director Peter Orszag further describes this section as a pilot in a blog post released on Oct. 7, stating:

The agencies participating in this initial effort will serve as demonstration projects through which we can test approaches to improve program effectiveness and efficiency, share best practices, and further improve performance. After assessing this initiative in FY2011, the Administration will be better positioned to implement government-wide evaluation metrics.

The limited scope and voluntary nature of this part of the initiative may help alleviate some problems encountered in the past by getting the Obama administration's performance measurement effort off on the right foot. Overall, this memo gives agencies more of a central role in developing their own evaluation systems and has a more collaborative tone than previous performance improvement efforts. Yet this is not a comprehensive plan to replace the PART, as both Orszag and Chief Performance Officer Jeffrey Zients have hinted that more is coming at some point in the future. There are currently no details regarding when additional aspects of the Obama administration's overhaul of performance systems will be released.