Cuts Proposed
National Research Council notes inconsistencies in PART use
The Bush administration recently proposed
eliminating or decreasing the budgets of 151 federal
programs in fiscal 2009, citing poor performance and
redundancy, and basing its recommendations in part
on results from its Program Assessment Rating Tool
(PART). President Bush said the proposed cuts
would save an estimated $18 billion and Jim Nussle,
Director of the Office of Management and Budget
(OMB), was quoted by GovExec.com as saying the
targeted programs "are frankly just not achieving the
results that they need to achieve." Congress still must
weigh in on the recommendations. Of 141 cuts
proposed for fiscal 2008, lawmakers enacted just 29.
About 70 percent of this year's proposed changes
were also part of fiscal 2008's budget
recommendations.
Meanwhile, a recent report from the National
Research Council (NRC) criticized how OMB applies
PART. In a review focusing on the use of PART to
evaluate research programs at the Environmental
Protection Agency (EPA), an NRC committee said
OMB places too much emphasis upon "ultimate
outcomes" and applies the tool inconsistently across
agencies. The NRC report indicated that OMB's
metrics to evaluate federal research programs were
inadequate, focusing primarily on process
efficiency. "Ultimate outcome-based efficiency metrics
are neither achievable nor valid," and several other
metrics could be used to measure the process
efficiency of research programs, noted NRC. The
report said OMB doesn't focus on investment efficiency
metrics and warned that evaluation of
research "should not over-emphasize efficiency,"
because it is only one part of a proper evaluation and
because the primary goal of research is the
development of knowledge.
The NRC report indicated that OMB had rejected
metrics proposed by EPA that were similar to those it
had approved for use by other agencies, and had
encouraged EPA to apply earned value management
(EVM) to its programs when no other agency has used
it to measure basic research. NRC urged OMB to
apply the same efficiency standards to all agencies
and to conduct oversight and training programs to
make sure budget examiners apply PART consistently
and equitably.
"Both of these stories underscore the
importance of government evaluation policies and the
care with which they must be formulated and applied,"
says George Grob, President of the Center for Public
Program Evaluation and a consultant to AEA's
Evaluation Policy Task Force (EPTF). "AEA needs to be
at the table when such policies are made." AEA
President and EPTF Chair William Trochim
agrees. "All evaluators should be concerned when
respected organizations like the NRC raise critical
questions about how evaluation is being
accomplished and utilized." said Trochim. "Members
of AEA's EPTF will be meeting with officials at OMB
and others to discuss the concerns raised by the NRC
and other organizations and consider how AEA might
enhance the quality and use of evaluation in the
federal government."
Full-Text Access
Silent Auction