In response to a January 2018 request from NOAA Fisheries’ Northwest Fisheries Science Center, the ISAB reviewed the document A Power Analysis of Two Alternative Experimental Designs to Evaluate a Test of Increased Spill at Snake and Columbia River Dams, Using Smolt-to-Adult Returns of Anadromous Salmonids (January 2018 draft). NOAA’s analysis considers two general experimental designs: (1) a before/after design for which there is no variation of spill level within the prospective years and (2) a block design that includes variations between two spill levels during prospective years. NOAA Fisheries requested the ISAB’s review to help inform their analyses and recommendations regarding potential future operations at the Columbia River System mainstem hydroelectric projects.
The ISAB found that NOAA’s analysis is a standard assessment of the power to detect effects and appears to be structured appropriately. The methodology and conclusions are conceptually sound, and the block design provides advantages to a before/after study. The key advantage to the block design is that high year-to-year variation is controlled for by conducting both spill regimes in the same year. However, the advantages are somewhat tempered because of several sampling and estimation issues. In addition, the success of the proposed experiment may depend on a number of factors including the availability of sufficient water and tagged fish to actually implement the experiment, the assumption that past fish behavior (in the retrospective years) is indicative of what will happen under the new spill regime, and other operational issues that would need to be resolved before an actual experiment is implemented.
These operational issues include:
- the expected improvement in smolt-to-adult returns (SARs) associated with particular spill levels
- the impacts of low and high flow years on the study’s implementation
- whether the proposed spill regime is equally beneficial for fish of all sizes/ages when they start their migration
- impacts of the proposed spill regime on fish travel speed
- effects of new spillway detectors on the study design
- identification of adaptive management triggers, and
- the experiment’s effects on migrant survival when only half of the year is at higher spill levels compared to a full year at higher spill levels.
So, while a theoretical implementation (i.e., NOAA’s block spill paper) may show high statistical power, an actual implementation may have less power because of these problems. These issues and several topics that should be explored to strengthen NOAA’s analysis are described in the full report.
