The Independent Scientific Review Panel (ISRP) recently completed a review of 540 proposals submitted to the Northwest Power and Conservation Council to implement the Council’s Fish and Wildlife Program funded through the Bonneville Power Administration. The ISRP understands that the Council is now in the early stages of developing a review process for the next project solicitation. With this memorandum the ISRP provides suggestions on the next review process.
Background
The ISRP was created by the 1996 Amendment to the 1980 Northwest Power Act to provide scientific peer review of projects funded by the Bonneville Power Administration to implement the Northwest Power and Conservation Council’s Fish and Wildlife Program. Together with the Council, the ISRP has contributed to developing a project review process with a uniform proposal format, standard review procedures, and evaluation criteria. The intent is to have a transparent recommendation procedure that leads to the selection of projects that, when executed, will fulfill the goals of the Fish and Wildlife Program and restore fish and wildlife populations impacted by the hydroelectric system developed in the Columbia River Basin. The project solicitation process has evolved over the last decade to better meet the needs of project sponsors, the Council, and the Program.
The ISRP has now completed four reviews of all the ongoing and proposed projects funded through the Council’s Fish and Wildlife Program. An annual review of all projects was conducted for FY 1999 and 2000, and again for a multi-year solicitation (FY 2007-09) in 2006. For FY 2001-03, the Council and BPA adopted an alternative review and selection process in response to ISRP recommendations from the earlier annual reviews to better integrate the geographic site of projects, include site visits, presentations by sponsors, and responses by sponsors to preliminary ISRP review. This process was referred to as the Rolling Provincial Review. This review involved solicitations and review of projects from adjacent ecologically similar subbasins staggered over 2 ? years. This process is described in the Council’s 2000 Fish and Wildlife Program.
The subbasin planning that followed the completion of the first cycle of staggered Provincial reviews disrupted the sequence of the reviews. The Council undertook the development of subbasin plans for some 58 of 62 subbasins in the Columbia River Basin. These subbasins are aggregated into 11 Provinces. The ISRP, ISAB, and Peer Review Group members reviewed the subbasin plans for the Council, and the plans have subsequently been adopted as elements of the Council Fish and Wildlife Program. The subbasin plans are intended to form the basis for prioritizing Fish and Wildlife Program strategies within subbasins — although the ISRP found that justification for many of the recent FY 2007-09 proposals lacked a thoughtful use of subbasin plans.
Because projects that were reviewed in the early cycles of the rolling provincial review had not been reviewed for many years, and because the Council wanted to employ the newly adopted subbasin plans in the project selection and implementation process, the Council chose to employ a multi-year (2007-09) solicitation for the entire basin in 2006. Five hundred and forty proposals were received and reviewed by the ISRP.
The ISRP recommended in its 2005 Retrospective Report (ISRP 2005-14) and again in its Programmatic Comments to the 2007-09 Solicitation Report (ISRP 2006-4a) that future processes be modeled after the sequential multi-year provincial reviews, with potential alterations to more efficiently address program needs through targeted and topical (wildlife O&M, systemwide RM&E, lamprey, and such) solicitations. A staggered review process that provides for site visits, presentations, and response loops between sponsors and the ISRP improve the Fish and Wildlife Program because the ISRP gains a more thorough understanding of projects and projects are improved by incorporating the ISRP’s constructive suggestions.
In the 2005 Retrospective Report, the ISRP also recommended that alternative review paths be investigated for continuing and new projects. For example, long-term operations and maintenance projects could receive administrative review or programmatic review of common methods, other continuing projects could receive periodic scientific review for progress attained (with funding of non-performers discontinued), and new projects could be reviewed both technically and administratively for responsiveness to targeted solicitations. The annual review process might thus concentrate on new proposals and a subset of the continuing projects.
The remainder of this memo provides additional details from the ISRP on suggestions for implementing a return to a staggered project review. The rationale for these suggestions is thoroughly explained in the 2005 Retrospective Report. The ISRP’s experience in the FY 2007-09 process further validated our earlier observations and recommendations. The ISRP looks forward to working with the Council and its staff on developing future review processes.
Elements to Consider for Future Fish and Wildlife Program Project Reviews
- Establish separate solicitations (RFPs) and review tracks for new projects targeted to specific problems including systemwide information gaps or key limiting factors in a particular subbasin. These solicitations need to be linked to the objectives and strategies in subbasin plans, the Council’s research plan, and the Fish and Wildlife Program. These targeted solicitations should use specific criteria that allow the ISRP to add value to reviews by ranking or indicating relative priority of proposals for satisfying a specific program need. In addition, as stated by the Council in its FY 2007-09 recommendations, continue to solicit innovative projects. For all targeted solicitations, the Council might want to explore the use of pre-proposals to screen qualified proposals to be developed into full proposals. This approach was employed for the FY 1999 RFPs.
- For ongoing projects with large out-year obligations, return to multi-year, sequential reviews making use of the assessments, objectives, strategies, and prioritizations (if done) in subbasin plans. The initial rolling provincial review subdivided the Columbia River Basin into 11 provinces with approximately 62 subbasins plus a mainstem/systemwide category. For many of the salmon habitat restoration, wildlife habitat acquisition, and salmon hatchery production projects this geographic subdivision is likely suitable. In addition to sequential provincial reviews, reviews of basinwide topics such as widely distributed critical species (for example, lamprey, sturgeon, bull trout, and invasive species) and systemwide research, monitoring, and evaluation (including supplementation, and tagging and tag detection) could be conducted.
- The sequenced review of ongoing projects is intended to provide for a thorough review by the ISRP of programs that are multi-faceted in their activities and potentially involve sponsors from different institutions. Examples include the various strategies and actions in the Umatilla and Yakima River subbasins, where there are habitat improvement and fish production actions by different agencies (and in the case of the Umatilla River, water transfers from the mainstem Columbia River).
- The ISRP review process would include (a) preliminary examination of a proposal and past reports and monitoring and evaluation data; (b) a tour of past and proposed project sites; (c) presentation of the proposal (preceding or following the site visit depending on the review type and logistics) with an opportunity for questions from the ISRP; and (d) a preliminary ISRP review with a response loop to provide sponsors with the opportunity to incorporate ISRP suggestions. A three-day site visit schedule for an ongoing program might include Day 1 - program overview and proposal presentation, Day 2 - site tour, Day 3 – discussion and question and answer session. A written report by the ISRP would complete the review.
- In addition to the elements requested in the proposal form for the FY 2007-09 solicitation, the proposal form for ongoing projects should be revised to add sections requesting explicit presentation of the objectives proposed in the preceding solicitation cycles, what has been accomplished, what has not been accomplished, and how benefits to the focal species are being evaluated. In the current form, this reporting is requested under the Project History and Objectives, Work Elements, and Methods sections, but it became clear in the FY 2007-09 review that this reporting needs to be emphasized and requested in separate sections.
- A first step in initiating such a review is going through the projects funded through the FY 2007-09 solicitation and aggregating projects and topics with Council staff to sort projects to be reviewed by topic and by subbasin/province (geography). When reviewing projects in subbasins, the review should be structured to include a multi-disciplinary approach, integrating habitat and hatchery (production) projects, even when some of the production projects may have been reviewed as a topical subject. In this way the sponsors and the ISRP can realize how these efforts mesh to fulfill the strategies and priorities established in a subbasin plan. Topical reviews should group projects to address program-wide themes such as common monitoring and evaluation methods for supplementation projects.
- The reviews could begin in FY 2007 with projects (geographic areas – Umatilla?) that require actions to maintain funding in FY 2008-09 as conditioned by Council.
