Summary
The ISRP reiterates its support for a monitoring and evaluation plan for the Lower Columbia River Estuary. In its review of the original M&E proposal, the ISRP rated it fundable, contingent on a more comprehensive and detailed explanation of the proposed monitoring program and database. In its recommendations for the Lower Columbia River and Estuary provinces the Council supported the ISRP's recommendation by requiring that a more comprehensive description of the monitoring plan be provided as a condition of funding. Subsequently, on April 1, 2004, the Council requested that the ISRP review the Scope of Work for Habitat Monitoring and Water Quality Monitoring in the Lower Columbia River and Estuary.
The ISRP rates the Scope of Work for the Water Quality Monitoring as fundable subject to more detailed documentation of the sampling protocols (location, time, and methods), but does not recommend funding for the Scope of Work for Habitat Monitoring. Detailed comments on each Scope of Work are provided in the attached memo.
Memo (document ISRP 2004-9)
TO: Doug Marker, Fish and Wildlife Division Director, Northwest Power and Conservation Council
FROM: Rick Williams, ISRP Chair
SUBJECT: Lower Columbia River Ecosystem Monitoring and Data Management, Project 2003-007-00 (Proposal #30015)
The ISRP reiterates its support for a monitoring and evaluation plan for the Lower Columbia River Estuary. In its review of the original M&E proposal, the ISRP rated it fundable, contingent on a more comprehensive and detailed explanation of the proposed monitoring program and database. In its recommendations for the Lower Columbia River and Estuary provinces[1] the Council supported the ISRP's recommendation by requiring that a more comprehensive description of the monitoring plan be provided as a condition of funding. Subsequently, on April 1, 2004, the Council requested that the ISRP review the Scope of Work for Habitat Monitoring and Water Quality Monitoring in the Lower Columbia River and Estuary.
The ISRP rates the Scope of Work for the Water Quality Monitoring as fundable subject to more detailed documentation of the sampling protocols (location, time, and methods), but does not recommend funding for the Scope of Work for Habitat Monitoring. Detailed comments on each Scope of Work are provided below.
Habitat Monitoring SOW
The ISRP recommends that the Habitat Monitoring SOW for the Lower Columbia River and Estuary not be funded. The SOW, like the original proposal, is too vague and general to be reviewed for scientific merit and benefits to fish. It remains a plan to develop a monitoring program, providing very little detail on the sample design, the proposed classification system, habitat performance indicators, or methods of analysis. Specific descriptions of current M&E activities and critical knowledge gaps — the type of information expected to be provided in a proposal's literature review — are absent. Activities presented as proposed tasks, for example the review of the literature and development of sample design, should have been done in the course of proposal preparation.
It is unclear what critical unmet need the SOW is intended to address because it does not provide a clear background. The reader is not provided enough information to know whether this project would fill important knowledge gaps or would simply duplicate ongoing efforts.
In addition, the proposal continues to lack any discussion of how habitat monitoring will be linked to fish population monitoring. A more comprehensive discussion of monitoring should have been possible. The RME Plan (2003) is referenced, implying that the estuary is included in the RME Plan. However, the RME Plan does not include sufficient detail concerning monitoring for status and trend of estuarine habitat and other biological parameters to fulfill the needs of this SOW.
The proponents should develop a plan to co-locate "on the ground" sampling sites for the different components of the proposal, including monitoring of salmon populations. Part 1 includes monitoring of status of salmon populations in the title, but addresses only monitoring of habitat status. Part 3 deals with monitoring for status of invasive species. The information in long-term status and trend monitoring programs is greatly enhanced for statistical analyses and model building if sites for study of habitat, salmon populations and invasive species (and potentially other indicator variables) are co-located in so far as possible given sample sizes. Design of the three elements (status monitoring of habitat, salmon populations and invasive species) should be closely coordinated. Co-location of study sites can be accomplished using different sample sizes with the EMAP procedures for spatially balanced site selection as referenced in the RME Plan (2003).
Detailed technical comments follow:
Part 1 — Salmon Population/Habitat Status Monitoring
Part 1 refers to salmon population monitoring but there is no inclusion of fish in the monitoring tasks or protocols. This project proposes to develop draft monitoring protocols, including methods and metrics. The choice of protocols, methods and metrics should have preceded the proposal and been described in the proposal text. This comment applies to Tasks 1.A, 1.B, and the first half of 1.C. For example, the objective for Task 1.B to "develop scope for pilot monitoring" is unclear. To say that the scope of the pilot study is to be defined after metrics, sampling strategies and protocols are developed is to defer work that should have been done in advance of the proposal. The second half of 1.C "implement monitoring", and 1.D. "evaluate monitoring results" are the logical tasks for the project.
Overall, the description of the approach to the Part 1 tasks is very tentative and to be determined at some future point. If the investigators have to interview habitat experts to get their best judgments on monitoring needs, indicator species and possible methods, it raises the question as to whether they have the necessary expertise to conduct the research. "Work with researchers that may be contracted to build on existing models", etc. shows a similarly tentative and uninformed approach. The description of year 3 activities is also quite general and uninformative.
Sampling Design
The SOW vaguely hints that a probabilistic sampling plan will be implemented, but detail that would allow the Panel to judge the validity of the plan is not provided. Methods of statistical analysis are not described, so it is not possible to judge if the proposed data collection will support analyses. There is some language that leads the ISRP to infer that a stratified random sample based on habitat types may be used. However, there is no assurance that definition of habitat types will result in strata with boundaries that do not change over time. Fixed boundaries between strata are not a necessary condition for analysis of "on the ground" (on the water) data collected in long-term monitoring programs. However, when boundaries between strata are moved, unequal probability sampling occurs, requiring weighting of data in analyses. Under these conditions statistical analysis becomes increasingly complex over time. For example, it is not simple to test for differences between means of an indicator variable from different areas (times) with data that are unequally weighted. Under these conditions a standard t-test does not meet the required assumptions to be valid. As a further complication, the theory of geostatistical analyses of unequally weighted data (i.e., unequal probability sampling) is not sufficiently developed to be available in standard textbooks and statistical software.
In another part of the proposal, the sponsors state that "The Estuary Partnership's LCRE Habitat Monitoring Program is consistent with the RME Plan (2003) and, in fact, the LCRE may be treated as a pilot monitoring subbasin (p.76-88, RME Plan 2003)." Pages 76-88 of the RME Plan (2003) deal with implementation of the so-called EPA EMAP (or Oregon Plan) for probabilistic site selection in "linear" tributary habitat. Hence in the present proposal for monitoring status and trends in the estuary, we are led to believe that the EMAP procedures for selection of sites in a two-dimensional area, i.e., the estuary, may be used to select sites for monitoring within strata. For example, Stevens and Olsen (2003) have used the EMAP procedures for selection of study sites in a two-dimensional reservoir. However, the sponsors do not provide details for site selection in the estuary, in stark contrast to details provided in the RME Plan (2003) for study of status and trend in tributary habitat.
It is not possible to judge the scientific merits of the proposed status and trend monitoring plan, because, as indicated above, there are considerations left unaddressed if strata are to be based on habitat types. Insufficient information is provided concerning application of an EMAP spatially balanced site selection procedures within strata, if, in fact, that is what the proponents are planning.
The Panel judges that a portion of Part 1 could be recommended to the Council if it were fully developed for review. Habitat status monitoring could be conducted using the EMAP spatially balanced/equal probability sampling procedure to select study sites, as implied by reference to the RME Plan (2003).A simple stratification of the estuary with a small number of strata based on, for example, river mile may be the best long-term plan for "on the ground" monitoring of status and trend of habitat, water quality, contaminants, and other biological parameters. The proponents should rely heavily on the theory for "domain estimation" in anticipated statistical analyses (Cochran 1965) and unequal probability estimation for combination of data across strata. We also comment that it is usually not possible to apply existing monitoring programs to the development of statistically valid inferences on status and trends of habitat and other biological parameters for large areas such as the Columbia River estuary. While it may sound like a good idea, the necessary requirements for valid statistical inference are almost never present. It is usually better to make a fresh start and design the status and trend monitoring from scratch.
One characteristic of the EMAP procedure with equal probability sampling is that a very large list of sampling sites can be created such that any fixed number of sites from the top of the list represents a spatially balanced sample. Further, the list can be used to populate any strata that are created later with a spatially balanced sample, including population of a pilot study area with study sites. Sample sizes within strata can easily be increased or decreased using the list. Data collected in the pilot study area can potentially be part of a longer term monitoring program while sites not visited are considered as "missing data" during the program development. It is important that the pilot study area not turn into an "index" area that is not part of a monitoring program that allows valid statistical inferences to be made to larger areas of the estuary. Tasks 1.A, 1.B, 1.C, and 1.D might then be implemented as planned.
Part 2 — Ecosystem Monitoring
Part 2 is also described in very general terms and is unclear with regard to how what is being proposed is different from what is already being done. What other entities are synthesizing GIS data? What is the added value of this proposed effort? The proposal is tentative as to whether data gaps exist that the project will fill, saying only that they likely do.
Reviewers inferred that Part 2 relates to a compilation of existing mapping results, followed by identification and filling-in of data gaps. Again there was no reference at all to salmon usage data. The reference to models in the sixth bullet seems to refer to habitat requirements, presumably for salmonids, but this is not described further.
The ISRP understands the need for Tier 1 monitoring and classification of aquatic habitats and geomorphology of the lower Columbia River by various model based analyses. The SOW should be able to at least define the general form and scope of the classification system. The SOW states that it will draw on existing classification approaches. A description of these approaches and how they may or may not be relevant to the Columbia River would help convince the reviewers that the necessary background study has been undertaken. Will the classification be hierarchical, as most habitat classifications are? What will the habitat hierarchy consist of? What metrics are likely to represent habitat conditions of each hierarchical level?
Use of Tier 1 monitoring results in GIS data layers is an important and necessary part of any comprehensive monitoring program. One of the primary goals of Part 2 is that "the classification system will meet current and future needs of all sampling and monitoring efforts of habitat, water quality, contaminants and biological efforts." Classification systems (models) will evolve with time and data layers created in, say 2006, will be reworked as new knowledge is gained. The classification model to be used in 2025 probably has not yet been conceived; at least we can hope that models will continue to be improved in the future. As stated above, statistical analyses for detection of trend or change in status become very complex if based on classifications that change over time, including reclassification of old data, say 2008 data, by new models.
Part 3 — Invasive species monitoring
Task 3 has an informative introduction and does present a good case for an important concern, but the funding is trivial and clearly is not a major portion of this proposal.
It is not clear whether information on presence and distribution of invasive species already exists or is a critical need. For example, the statement "Invasive species are known to be increasing in a real extent and in diversity within the system," suggests that presence and distribution are already known. The proposal refers to programs that are monitoring invasive species — but other than saying "we feel there is utility in summarizing the information that is currently available", the proposal doesn't justify this activity. It isn't clear whether the proposed activities will replicate ongoing activities or add new and needed ones.
Some of the tasks, such as Subtask 3.A.2., sound as if they are going to coordinate with ongoing activities. If this is the case the nature of the coordination should be explained. Other tasks, such as Subtask 3.A.3. are too generally worded to understand their specific meaning.
The deliverables for the research tasks should be described more specifically. They are presently described quite tentatively as "expected products."
Water Quality Monitoring
The ISRP finds the Water Quality Monitoring SOW to be fundable subject to a more detailed documentation of the sampling protocols (location, time, and methods). Overall, this SOW was professionally written, informative, and worth supporting. It was better organized and provided more specific details on performance measures, sampling sites and methods of sampling and analysis than did the Habitat Monitoring SOW. The Water Quality Monitoring SOW identified the major uncertainties concerning contaminant effects on estuarine organisms and attempted to shape the objectives around these uncertainties. It provided a better review of background knowledge to support the objectives and methods than the Habitat SOW. The sponsors made an admirable effort to include federal, state, and City agencies as participants in development of the plan.
A more detailed description of the sampling protocol " sample design and collection methods " is needed. Composite sampling does not constitute a statistical design, and it will be compromised by variation in flow rates. The description should include the rationale behind using NOAA Fisheries sampling sites. It should also provide evidence of the added value of the proposed sampling.
The SOW relies heavily on models of various kinds for describing contaminant flux, uptake, effects, and risks. The sponsors refer to a "semi-quantitative model" that apparently is intended to bridge the gap between a conceptual model and a fully quantitative model, but they fail to clearly define what they mean by a "semi-quantitative" model. A more serious concern, however, is the proposal to link, via a model, contaminant body concentrations to growth, survival, and fecundity so as to predict toxicant impacts on population dynamics. This exercise is by no means routine and is bound to be fraught with high uncertainty. The sponsors should have provided an objective discussion of the possibilities and difficulties of this exercise and discussed how they would deal with uncertainty issues.
Detailed technical comments follow:
Task 1 — Salmon Exposure and Ecological Risk Model
Task 1 presents a logical process for development of a conceptual model and review of data, progressing to the development of the risk model. Each task has a timeline and deliverable, with identification of the partners working in these issues and areas.
However, there are several areas under Task 1 for which more information should be provided. What is the source of information for the conceptual model, for example the "likely effects of exposures on salmon health and survival"? Will there be enough information to get to the end point ? assessing the effects of exposure on salmon? Will enough be known about effects of exposure to model consequences on populations? Details on monitoring methods are sparse and should be expanded. Deliverables should be more specifically described.
Task 2 — Water quality monitoring in the lower Columbia River Basin
Task 2 describes the development of sampling locations and methods. It is much more specific as to what has been done and why the proposed work is needed. It provides a rationale. However, the goal statement should be clarified. What does it mean (in the context of this proposal) to say "linkages between the presence of toxic contaminants and salmon populations with respect to the FCRPS"? It is not clear how water quality will be linked to the FCRPS.
The rate of sampling seems limited and the numbers of sites in the actual estuary are very limited. The absence of estuary sampling requires some response from the authors. Additionally, more detail as to whether existing sampling sites will be adequate to the task is needed. The SOW should also be more specific about the value added from this effort; e.g. if the suite of sampled chemicals is known to be present in the Columbia River, what will be gained form further sampling?
Task 3 — Juvenile salmon synoptic monitoring
Task 3 addresses salmonid sampling and provides a very interesting project description. The sampling basis is not explained (i.e., how variable are the results between individuals etc.), but may be limited by the cost of analyses. If this is the case it would be helpful to have the text explain the limitation. Task 3 also leaves open whether other researchers are continuing to work on effects of these pollutants on salmon. Reviewers recognize that this topic is beyond this monitoring proposal, but this may be an area of future interest to the Council.
Task 4 — Summary report for the lay audience
Task 4 is highly appropriate given the interest that is likely to develop from this monitoring. It would be good if the authors keep in mind the policy audience as they write this report. A focus on how habitat, water quality, and salmon population health relate to policy controls would be helpful in guiding needed policy change.
Literature cited
Cochran, W.G. 1965. Sampling Techniques, Second Edition. John Wiley & Sons, New York.
RME Plan.2003.Research, Monitoring & Evaluation Plan for the NOAA-Fisheries 2000 Federal Columbia River Power System Biological Opinion. Draft September 11, 2003. NOAA Fisheries, Bonneville Power Administration, U.S. Army Corps of Engineers, and Bureau of Reclamation.
Stevens, D.L., Jr., and Olsen, A.R. 2003. Variance estimation for spatially balanced samples of environmental resources. Environmetrics 14: 593-610.
[1] www.nwcouncil.org/media/16871/2003finalrec.pdf - Columbia Estuary Issue 3, page 14.
