Since its inception in 1977 from an amalgam of federal authorities, the U.S. Department of Energy (DOE) has administered numerous programs aimed at developing applied energy technologies. In recent years, federal oversight of public expenditures has emphasized the integration of performance and budgeting. Notably, the Government Performance and Results Act (GPRA) was passed in 1993 in response to questions about the value and effectiveness of federal programs. GPRA and other mandates have led agencies to develop indicators of program performance and program outcomes. The development of indicators has been watched with keen interest by Congress, which has requested of the National Research Council (NRC) a series of reports using quantitative indicators to evaluate the effectiveness of applied energy research and development (R&D).



The first such report took a retrospective view of the first 3 years of DOE R&D programs on fossil energy and energy efficiency. The report found that DOE-sponsored research had netted large commercial successes, such as advanced refrigerator compressors, electronic lighting ballasts, and emission control technology for flue gas desulfurization. However, some programs were judged to be costly failures in which large R&D expenditures did not result in a commercial energy technology. A follow-up NRC committee was assigned the task of adapting the methodology to the assessment of the future payoff of continuing programs.



Evaluating the outcome of R&D expenditures requires an analysis of program costs and benefits. Doing so is not a trivial matter. First, the analysis of costs and benefits must reflect the full range of public benefits that are envisioned, accounting for environmental and energy security impacts as well as economic effects. Second, the analysis must consider how likely the research is to succeed and how valuable the research will be if successful. Finally, the analysis must consider what might happen if the government did not support the project: Would some non-DOE entity undertake it or an equivalent activity that would produce some or all of the benefits of government involvement?



This second report continues to investigate the development and use of R&D outcome indicators and applies the benefits evaluation methodology to six DOE R&D activities. It provides further definition for the development of indicators for environmental and security benefits and refines the evaluation process based on its experience with the six DOE R&D case studies.



Table of Contents


Front Matter
Summary
1 Introduction
2 Results of Applying the Methodology
3 Methodology for Prospective Evaluation of Deparment of Energy
Programs
4 Expert Panel Process
5 Conclusions and Recommendations
References
Appendix A PART Assessment Questions
Appendix B Committee Biographies
Appendix C Statement of Task
Appendix D Letter Report
Appendix E Committee and Panel Activities
Appendix F Guidance on Prospective Benefits Evaluation
Appendix G Information to Be Requested of the Department of Energy
Appendix H Report of the Panel on DOE's Integrated Gasification
Combined Cycle Technology R&D Program
Appendix I Report of the Panel on DOE's Carbon Sequestration
Program
Appendix J Report of the Panel on DOE's Natural Gas Exploration and
Production R&D Program
Appendix K Report of the Panel on DOE's Distributed Energy
Resources Program
Appendix L Report of the Panel on DOE's Light-Duty Vehicle Hybrid
Technology R&D Program
Appendix M Report of the Panel on DOE's Chemical Industrial
Technologies Program