The purpose of this task is to review the status of available information for fish and wildlife projects in 2007 fish and wildlife project proposals (proposals) and in Pisces, and to recommend whether a larger task is advisable to develop ways to improve the cost effectiveness of the project selection process. Pisces is a computer program developed by BPA’s Fish and Wildlife Division to improve management of Fish and Wildlife projects, and to improve the ability to report information about the Fish and Wildlife Program.
The objective is to see if costs of different strategies can be approximated in a way that could support cost-effectiveness analysis or cost benchmarking. Cost-effectiveness analysis compares the costs of alternatives, usually on a per-unit basis, to determine which is least costly, or to obtain an objective at least cost. Cost benchmarking uses costs of existing activities to develop cost standards or guidelines to help judge the reasonableness of future proposed costs.
The IEAB has examined information available in a subset of proposals and examined information for ongoing projects available in Pisces. In general, the quality of information available for cost-effectiveness analysis or cost benchmarking is variable among proposals, and in Pisces, among the standardized work elements and metrics. Work elements are standardized definitions of work activities. Metrics are standardized quantitative measures of accomplishment associated with some work elements.
Among the proposals, there is great variation in the types of projects, their objectives, and local conditions that affect activities and their expected costs. For most work elements, no metric is reported. For some projects, BPA pays for a fraction of the total cost and the proposals do not show the other cost shares. For most work elements there are several types of activities and costs reported across proposals. Some of the metrics in the proposals are not clear in terms of what is being reported. It appears that some work elements could have a metric but none is reported.
Some types of work elements and metrics – miles of fencing, acres of vegetation removed, or smolts produced, for example, lend themselves to cost comparisons, but for typical questions about benchmarking and cost-effectiveness, there are still project-specific and site-specific conditions that should be considered. More analysis of these conditions might lead to a formula where expected costs can be estimated as a function of site conditions. For other work elements, some disaggregation beyond the existing work elements may be needed to isolate the costs of measurable activities within the element. It may not be possible to specify a standard metric for cost-effectiveness analysis for some work elements such as environmental compliance, data collection, analysis or reporting, but it will still be useful to show the share of total project cost devoted to these types of activities. For some applications, these "foundational" or joint costs may need to be reviewed and possibly allocated among other metrics.
Pisces improves on the proposal information in several ways. The proposals provide information on planned, rather than actual, costs and metrics. Pisces provides actual costs and metrics for ongoing projects. Some of the information in the proposals is not checked or verified. Some errors and inconsistencies are corrected before actual project information is entered into Pisces. BPA intends that Pisces will improve on the quality of information provided in project proposal forms (BPA 2006).
Pisces is a project management tool and is not designed for economic analysis. Pisces information will be helpful, but more detail will often be required for useful analysis. As with the proposals, there is a large range of activities within work elements, so additional information about the actual work and costs should be reviewed. The IEAB and Pisces developers have an ongoing dialogue to help improve the quality of economic information contained in Pisces.
For most projects, some of the important cost, engineering, and site-specific information is not provided in the proposals or Pisces. In particular, more information about the expected life of improvements and the duration of benefits would be useful for basic economic analysis. The share of cost that is a long-term investment, the expected life of these investments, and the expected timing of benefits in the future, should be provided if available. It is our understanding that contractors will provide this information for Pisces at the contracting phase when statements of work are being developed.
Metrics should be tied directly to proposal objectives. The number of fish produced is not provided as a metric for habitat, and the disposition of fish production is not provided as a metric for hatcheries. The IEAB recognizes the difficulties and uncertainties associated with estimating and providing this type of information. Still, we continue to support efforts to improve information about the amount, types and disposition of fish and wildlife produced.
Based on this preliminary review, the IEAB recommends that a larger task be developed to identify opportunities to increase the value of proposals and Pisces information for cost-effectiveness analysis and cost benchmarking. In particular, the IEAB would:
- Attempt to develop some cost data for cost-effectiveness analysis and benchmarking based on all data provided by Pisces, augmented by current and past proposals, and report on the potential use(s) and quality of the cost information developed.
- Investigate cost data that may be available from other public and private sources and determine if it could be used for cost-effectiveness analysis or cost benchmarking of fish and wildlife projects.
- With BPA, review the list of work elements and their metrics to see if more detailed work element categories and measures can be recommended, and continue working with BPA and Pisces to increase the value of information provided by proposals and Pisces for economic analysis.
