Background
At the request of the Council, the ISRP provides this review of the Shoshone Paiute Tribe's Monitoring and Evaluation Plan, which was generated in response to ISRP comments and Council recommendations from the Upper and Middle Snake provincial review. In its final June 7, 2002 report for those provinces, the ISRP rated the Shoshone Paiute Tribe's proposal Enhance and Protect Habitat and Riparian Areas on the Duck Valley Indian Reservation as "Not Fundable" because the proposal lacked quantifiable biologically measurable objectives, the methods to achieve those objectives, and a scientifically sound monitoring and evaluation plan to determine the progress toward meeting those objectives. In September 2002, the province group and Council supported the continuing operation and maintenance of the project to maintain past investment. The Council, however, advised the sponsor that the planning, design, and construction and implementation funds requested cannot be provided unless and until a monitoring and evaluation (M&E) plan is reviewed and found acceptable by the ISRP. The Council received a revised M&E plan submittal on August 18, 2004.
Review Summary
The overall recommendation of the ISRP is that the "Monitoring Strategy for the Duck Valley Indian Reservation" by Hillman and Dykstra is fundable as part of Project #199701100. Some issues remain regarding the application of the monitoring strategy and are discussed below. A summary of the plan was presented as part of the Owyhee Subbasin Plan and, in general, received a favorable review by the ISRP/AB team. We recommended that the section could be augmented by a better description of the monitoring and evaluation logic path and better identification of the specific terrestrial and aquatic variables to be monitored and evaluated, including data collection protocols. The sponsors have provided the requested information in the form of the Hillman and Dykstra report.
Funding has been provided for a number of years in support of projects aimed at examining the status and distribution of redband trout and their associated habitats on the Duck Valley Indian Reservation (DVIR). The ISRP has offered specific review comments and suggestions for improvement to the ongoing aquatic assessments. The report by Hillman and Dykstra is a good and logical step forward from those earlier discussions. A notable omission from the DVIR M&E Strategy report, however, is any description of the stream and redband trout monitoring data collected over the last few years under ISRP-reviewed, Council-supported, and BPA-funded projects, and how, if at all, that data will be incorporated into this proposed M&E program. The Hillman and Dykstra report treats the proposed DVIR M&E strategy as a de novo program, rather than as a continuation and integration of earlier efforts.
A continuing primary concern of reviewers is at the larger scale of "rolling up" data from multiple subbasins to provide meaningful data and inferences to larger areas. The Owyhee Subbasin Plan made a commitment to coordinate aquatic monitoring activities among subbasins. This document continues to promote such cooperation, e.g., indicator variables are reported to be consistent with indicators identified in the Action Agencies/NOAA Fisheries Research, Monitoring and Evaluation Plan, the Upper Columbia Monitoring Strategy, and the WSRFB (2003) monitoring strategy. We cannot overemphasize the importance of cooperation and coordination as the region moves toward more consistent and comparable monitoring procedures.
The proposed M&E is thorough and comprehensive, perhaps more so than is needed to understand redband trout abundance, distribution, and trends and riparian stream conditions in the DVIR. Future funding levels may constrain this ambitious M&E program and dictate a smaller more economical set of metrics for annual and long-term monitoring. Similarly, some changes and compromises will be necessary in the monitoring strategy for the DVIR as PNAMP, CBFWA, and other cooperative monitoring efforts are developed, including consistency with results of the Action Agencies' pilot projects for monitoring of status and trends of aquatic resources in the John Day, Upper Salmon, and Wenatchee subbasins. Project sponsors should expect to need to prioritize among monitoring activities. They need a definite budget to help focus this exercise.
Specific Review Comments
One technical error is carried forward in the monitoring plan and will be easy to correct. Sites that are in areas that are physically inaccessible or cannot be accessed because of landowner denial generate "missing data." Areas with missing data must be clearly documented. No design based (direct) scientific inferences can be drawn to areas that cannot be accessed. The inaccessible sample sites cannot be replaced by selection of alternatives from an "oversample." We grant that more precise estimates can be made to accessible areas by increasing the sample size, but the design-based inferences do not apply to the inaccessible areas. We also grant that there may be acceptable scientific methods to make model based inferences concerning the inaccessible areas, e.g., multiple regression estimates of the "missing data" if adequate variation exists in the accessible areas.
Unfortunately, the plan does not include monitoring for status and trends of upland terrestrial habitats and species. In fairness to the sponsors, there has been little progress within the Columbia Basin for development of cooperative plans for monitoring of status and trend of terrestrial habitat and species. We grant that the plan includes monitoring of presence and abundance of Columbia spotted frogs and yellow warblers in riparian habitats on the DVIR, and we strongly support this effort.
We were impressed to see, for the first time in our reviews, the use of spatially balanced sampling sites selected by the EPA-EMAP generalized random tessellation stratified design (GRTS). Use of this design is state-of-the-art probabilistic site selection for monitoring studies. Section 3 of the document that describes essential elements of monitoring contains a precise and clear definition of terminology and a good discussion of sources of error.
Reviewers noted some problems with the status/trend monitoring section as it would be applied to the DVIR. A plan like this should endeavor to use only indicators that are effective in obtaining goal-oriented information and make sense biologically and fiscally. The proposed plan makes no effort to justify its indicators, by either documenting how they are proven in previous studies or discussing how they have real promise. It is unlikely that the proposed use of fish aging - apparently for all species collected - is appropriate. Aging is often difficult to do accurately, is relatively costly, and its need, especially for non-salmonids, is not justified by the plan. Also, the routine collection and analysis of macroinvertebrates as specified has dubious value because a rigorous program is more expensive than monitoring of the fish themselves, and the kicknet sampling specified for the DVIR probably would not stand up to serious scrutiny.
The plan seems to weaken in its effectiveness monitoring section. It would monitor three sets of projects - spring fencing, some road modifications termed "reclamation", and work along 3.5 miles of the East Fork Owyhee River. The monitoring plan might be reasonable for the latter, which is yet to be initiated, but it is not for the other two projects. For example, the road "reclamation" was done in 2002/03 and there was nothing in the proposal that gave even a slight indication that future monitoring could be expected to detect any reduction in sediment delivery to streams.
