At the U.S. Army Corps of Engineers and Council's request, the Independent Scientific Review Panel (ISRP) reviewed the document "Statistical Design for the Lower Columbia River Acoustic-Tag Investigations of Dam Passage Survival and Associated Metrics". This project is proposed for implementation through the Corps' Columbia River Fisheries Mitigation (CRFM) Program, specifically the Anadromous Fish Evaluation Program (AFEP). ISRP review of projects under this program was directed in 1998 by U.S. Congress Senate-House conference report for the fiscal year 1999 Energy and Water Development Appropriations bill. The ISRP's review responsibilities are also incorporated in the Council's 2009 Fish and Wildlife Program.
The Corps and staff from the Northwest Power and Conservation Council identified this proposal for ISRP review because the proposed survival model and experimental design will be used to measure dam passage survival in accordance with the 2008 NOAA Fisheries Biological Opinion Juvenile Salmon Dam Passage Performance Standards and to assure compliance with the Columbia Basin Fish Accords. The project's purpose is to develop a survival model and experimental design for estimating survival through the Lower Columbia River dams toward meeting the 2008 Biological Opinion performance standards. Further, this project represents the Corps' efforts to develop a standardized plan of study, intended to apply sound scientific principles to implementation of AFEP-funded survival studies. Moreover, although developed for simultaneous implementation at the three lower river projects, the basis for the design is not constrained to those projects.
The ISRP found the proposal meets scientific review criteria of sound science, benefits to fish and wildlife, clearly defined objectives and outcomes, and provisions for monitoring and evaluation of results. The ISRP review states that the proposal is a thoughtfully prepared plan to evaluate how well the structural and operations improvements mandated for the Lower Columbia River (John Day, The Dalles, and Bonneville) projects are meeting the 2008 FCRP BiOp and Columbia River Fish Accords survival targets for yearling and subyearling Chinook and Steelhead. The supporting material, providing details of the statistical design and analysis, is comprehensive and useful. The survival model is grounded in standard statistical methods and uses advances that have recently appeared in the literature. The authors have sought outside advice, have done preliminary experiments, have learned from those experiments, and have adjusted the protocols to reflect that experience.
The experimental design for this project is well reasoned, justified, and described. Procedures are given to assure quality control, to trouble-shoot problems that may occur with data collection, and to verify key assumptions. Experience to date with the juvenile salmon acoustic tagging system (JSATS) has been encouraging; nevertheless, the scale and complexity of the proposed project makes it probable that modification of the experimental design will be necessary during data collection.
The investigative team is relying on detection capabilities of fish implanted with JSATS rising to 90% or greater below Bonneville in this current year, and the sample sizes have been tentatively set on the basis of this assumption. There is also uncertainty concerning which of the detector arrays downstream of Bonneville will provide the best detection rates, and some trial and error evaluation will be necessary in an effort to "calibrate" the design. Detection rates reported in the statistical design, while much improved over those of previous years, are still in need of further improvements. The extra acoustic arrays that are planned should certainly improve the situation, but contingency planning for the case where detection rates remain 50%, or rise only to 60%, 70%, 80%, but not to 90%, would be wise. In the presence of reduced detection probability, the uncertainty of survival probability estimation increases, as do the sample sizes needed to counter the loss of statistical resolution.
The assumption runs throughout the proposal that "tagging effects" will be relatively small. There is literature showing that any handling of smolts, and implantation of acoustic tags in particular, can result in increased mortality. The project will assess departures from the assumption of "no tag effects," but assumes they will be small. Over the time frame necessary to migrate from the tailrace of McNary Dam to the Columbia River estuary, the cumulative effects of the acoustic tags may be larger than anticipated, perhaps large enough to obviate the advantages of the three-project design. As acoustic tags become smaller, carrying them should become less of a burden, but handling and implantation will remain issues. This may require modification of the experimental design after the initial results are evaluated. The plans for regularly evaluating model assumptions as the study progresses are essential.
The ISRP notes that there is legitimate debate about how to determine sample size. The important point is that it would be good to set the sample sizes large enough to be sure to detect differences from the BiOp survival rates with high probability. One should set the sample sizes large enough to ensure that large shortfalls are almost certainly detected and that smaller ones are detected with respectable probability. The ISRP believes an essential objective of this evaluation effort should be to estimate the true survival rates, whatever they are, rather than simply rejecting the null hypothesis of "no significant departure" from the BiOp standards.
In addition to addressing the BiOp standards, the proposal addresses other critically important issues. In particular, data about the specific routes juvenile salmon and steelhead take on their downstream migration will be collected, and the relative success rates from these different routes will be assessed. Such information makes it possible to parameterize the models required for making management decisions. Knowing how many fish take each route makes it possible to understand whether structural changes, such as modifications in the spillways and bypass systems, are improving fish survival, providing direction for changes that should be pursued or abandoned.
It is important that these data be preserved and made available at an accessible online site. In the long term, improvements in analysis of the data will come with improvements in physical filtering to separate noise from signal and in digital filtering to improve the sample size and to separate live fish from background noise. This will lead to a better understanding of fish behavior. In particular the 3-D data need to be protected to allow extracting information about routes fish take through projects. The COMPASS model was developed with the ability to take advantage of such data, and it is important that this ability be fully utilized in management experiments.
See the attached memo for the full technical review.
