|
News & Analysis | REG•WATCH Blog | Press Room
Wednesday, June 14, 2006
PARTial Responses
About the "Results Not Demonstrated" Score
Said Chairman Coburn:
Don’t believe the spin that “results not demonstrated” could mean that the program is either good or bad, we just don’t have enough information to tell. On the contrary - the “results not demonstrated” designation is a red flag marking a program so poorly conceived or directionless that unaccountability seems to have been built into it by design.
Hardly hype -- that's the definition OMB gives to the RND score. The RND score is given to programs that fail a couple of specific questions.
It is interesting to observe, however, that many of the programs scored RND otherwise score more highly than other programs in the section for producing results. Take a look at the FY06 PARTs:
- Seventy two of the 178 programs (40 percent) categorized as “Results Not Demonstrated” by FY 2006 had scores that, according to OMB’s own grading scale, would have been granted passing scores if not for failure on the specific RND-determining questions.
- Of these 72 programs, 12 should have received the high score of “Moderately Effective.” These 12 programs have higher scores for section 4 — the section that notionally measures actual results — than the average score for all programs actually rated “moderately effective.”
- Three programs — the Consumer Product Safety Commission, a USDA program for rural water treatment loans, and the National Credit Union Administration’s Community Development Revolving Loan Fund — actually scored above 75 percent for producing results (substantially higher than the 60 percent average for all programs rated “Moderately Effective”).
- Moreover, the remaining 60 of the 72 otherwise passing programs would have received the middle passing score of “Adequate” — again, if not for failing the specific RND-determining questions.
- More than half of them (31/60) scored higher for section 4 than the average section 4 score for all programs that actually received the score of “Adequate.”
- Almost as many (24/60) had overall scores that bested the average overall score for programs that OMB allowed to receive the “Adequate” score.
There is no explanation given for the weighting assigned to any of the particular questions or sections, nor for the absurd results once these weights are assigned. This inconsistency highlights an important point that emerged from our agency interviews as well. Implementation of the PART survey is highly dependent on the individual program officer at OMB, and working with different officers can not only completely alter the process by which the survey is completed but also the final rating for the program. The Government Accountability Office has also concluded that the PART gives a high level of influence to budget officers at OMB and leads to inconsistent application of the tool across the federal government.
Oversight
Again from Chairman Coburn:
One problem I have with this hearing, is one problem I have with this Congress, and that's: we're lazy. . . . How do we motivate Congress to do voersight? How do I motivate my peers to ... do the oversight...?
Admitting the problem of a lack of oversight, rather than insisting that Congress is actually doing its job, is the first step. It's great that Sen. Coburn admitted the lack of oversight. It's a problem that led Rep. Tanner to introduce a measure forcing Congress to do its job and hold oversight hearings whenever GAO and IGs produce reports identifying waste, fraud, and abuse. It's unfortunate that Congress would have to be forced to do its job on behalf of the American people.
Duplication
The kind of simplisticness in the PART approach to "uniqueness" was expressed in the testimony of Eileen Norcross:
Over the years Congress has created hundreds of programs addressing a single outcome. There are anywhere between 180 and 342 programs dealing with economic development in over 24 agencies. There are 44 job training programs in nine agencies; 130 programs serving at-risk youth, and 72 safe water programs, to name a few. Program duplication on this scale implies that Congress isn’t sure which programs are reaching their goals. It has no way of comparing programs with common outcomes.
Hold the phone here. "Program duplication on this scale" assumes we are really talking about wasteful duplication. The sheer number of programs addressing social problems can attest to the complexity of social problems more than waste. Take, for example, foster care. There are plenty of programs addressing the needs of abused and neglected children entrusted to the care of the states: Title IV-E entitlement, Adoption Assistance, Child Welfare, Medicaid, special education, community-based services, Chafee Independence Living, and so on. Lots of programs, but the sheer number does not mean that we are doing too much for abused and neglected children. The truth is, we are not doing enough.
Experience has shown over the years that programs held up to be duplicative are often programs that are geared to very different populations.
Additionally, having mulitple programs does not necessarily mean that we have multiple offices. Many programs may be administered by the same office.
And sometimes duplication is necessary. The Appalachian Regional Commission, for example, was created to duplicate some programs that were otherwise effective across the country because the structurally disadvantaged populations of Appalachia were not receiving the benefit of those programs. Occasionally, programs must target specialized populations -- take, for example, women's health programs, which address the problems that come from general health research programs that have long taken the male as the general standard -- and while they may have similar functions, they exist separately for a reason.
PART is just not sophisticated enough to recognize these issues.
Posted by Robert Shull
|