Background
At the request of the Council, the ISRP reviewed the Washington Department of Fish and Wildlife's proposal, Protocols for Counting Salmonids, Resident Fish, and Lampreys in the Pacific Northwest. This project would assemble and evaluate existing protocols for counting resident and anadromous fish (including lamprey) and recommend the most robust subset for consistent use across the Pacific Northwest. This proposal seeks funding of $72,870 out of the Fiscal Year 2003 place holder for data management. In addition, the proposal seeks $80,000 in Fiscal Year 2004 to complete the project.
In March 2002, the ISRP reviewed a preliminary Statement of Work (SOW) for this project, but found that the SOW did not provide adequate technical information and detail to allow the ISRP to evaluate the soundness of the proposed work. Consequently, the ISRP recommended that the SOW be expanded into a full technical proposal using the standard Council-BPA form to allow scientific and technical review. The proposal reviewed here was developed in response to this March 2002 ISRP recommendation.
Recommendation
The ISRP recommends funding of this project for production of the proposed database of survey methods with the information to be documented in the format of the Sponsor's Table 1. This aspect of the proposal is much needed and represents a necessary step toward documentation and collection of more accurate, precise and comparable monitoring data for the Columbia Basin. However, the proposal needs to be more specific about the procedures to be used.
Our primary reservation with the full range of activities is that the sponsor may be proposing more work than can be accomplished within the budget and time period indicated. Much of what is presented here will be difficult to accomplish, because there is little evidence that the documentation and data that these authors are looking for exists. For example, it will be difficult or impossible to assess the accuracy or precision of many of the protocols, because appropriate data do not exist. Section b, paragraph 1 refers to many types of fish counting protocols, but how will accuracy and precision of the counts be assessed and what determines compatibility, etc. Are there any systems where a variety of these protocols have been compared? However, if the sponsor can document the various procedures used in the Basin and how regularly these methods are applied, and begin to assess accuracy and precision of the methods, then the project would be very useful. If the information in Table 1 (Essential Elements of Protocols) can be recorded for each survey method, this would be a significant accomplishment for the proposed budget and time period.
The value of the expert panels and subjective ranking of protocols is questionable. Once an array of protocols is collated, how would a group determine the preferred protocols? This topic really gets to the CBFWA proposal (#35033) for a regional M&E project in that there will be several levels of data quality needed to answer various questions. To meet this need, there should be a set of counting procedures that are acceptable (based on some agreed set of standards) for counting programs with different types of objectives. The proposed database of survey methods and data to be documented in the format of the Sponsor's Table 1 would be valuable for CBFWA's project #35033.
One of the primary problems in improving monitoring methods is beyond the control of the sponsors of recent proposals, i.e., the "good old data sets that exist" and the reluctance of biologists and fisheries management agencies to change their monitoring methods. However, the region should go forward with pilot projects such as this one, #35019 (Chris Jordan, NOAA Fisheries, "Develop and Implement a Pilot Status and Trend Monitoring Program for Salmonids and their Habitat in the Wenatchee and Grande Ronde River Basins"), and #35033 (Collaborative, Systemwide Monitoring and Evaluation Program). The ISRP has confidence that projects such as these will demonstrate the long-term value of using probabilistic sampling procedures for site selection, common protocols for data collection when appropriate, and defensible statistical inferences to relatively large subbasins within the Columbia River Basin.
Peer Review Technical Issues (to be considered by the sponsor if this project is funded)
The "Essential Elements of Protocols" (Johnson et al. 2001:10) are lacking two essential elements:
- The Background and Objectives must identify the target universe for which fish counts are needed, e.g., the John Day River Basin for which an accurate, precise estimate of the number of adult hatchery steelhead strays is needed, or the Middle Fork of the John Day River for which the number of spring-summer chinook redds is needed.
- The Sampling Design must identify the sampling frame of possible study sites that exist in the target universe, e.g., all Order 1 sites of length 200 meters on the Middle Fork of the John Day River.
Objectives need to be more clearly indicated. Objectives are included in the running text but should be specifically listed. For example, a trend analysis over time may only need a consistent annual visual survey of a section of stream; but a stock assessment or research program needs quantitative data for species/age/sex and hatchery versus wild. Shouldn't the objective be to determine "robust" methods that meet a minimum data standard so that comparisons between streams and/or years have a level of accountability (or confidence) associated with them?
To "build on the data we already have" (section II, page 4) there are some steps needed in addition to documentation of the past methods and survey conditions. Specifically, it is necessary to:
- document past survey methods, effort, and basis of estimated values and document the percentage of research, monitoring and evaluation projects for "fish counting" that have full (without modification or updating), partial, or unavailable written protocols;
- establish appropriate data standards and protocols for use in an environment and for the specific program task;
- compare new protocols with past practices and develop a consistent time series of data;
- provide more detail on how protocols will be linked to the types of management projects and the selection thresholds to take a protocol to the next level; and
- develop more detail as to how these workshops will be conducted; i.e., if protocols have been evaluated prior to the workshops, what will the panels do, and what procedures will be followed to elicit consensus scientific rankings.
The abstract mentions counts of adults passing the mainstem dams as one of the proposed subjects for inclusion. However, issues associated with obtaining accurate and precise counts at dams were not fully developed in the proposal. The sponsor should carefully consider previous ISRP and ISAB comments on difficulties likely to be encountered with these counts, e.g., the fact that upstream dams often have higher counts than at downstream dams (see ISAB 99-2).
The ISRP recommends that the sponsor concentrate on the States of Oregon, Washington, Idaho, and the Columbia Basin. The comments about some other areas are not all correct, e.g., the Province of BC does not conduct any salmon or lamprey surveys and very few for steelhead or resident fish (except for interior BC fishes).
Minor comments:
- We were surprised that Dr. James Anderson's Second Tier database — Data Acquisition in Real Time (DART) was not mentioned as one of the databases that would benefit from the project.
- In the sentence, "This often requires substantial field-testing efforts that are now being conducted for some variables and key sampling protocols", do you mean "NOT being conducted"?
- Robust is a word that is used but not defined. Do you mean robust as in robust regression or in the sense of being insensitive to annual variability in survey conditions?
