|
Quality |
Broadening
Participation in Biological Monitoring: |
|
Rationale—Although it is possible that the dominant goal of a participatory monitoring project is simply to improve relationships among stakeholders, in almost all instances obtaining high quality data that is widely trusted is the central means of attaining the monitoring goals. Differences do exist in how important data credibility is to the users of the data. For instance, local commercial resource users who are collecting data predominantly for themselves might have less concern for outside validation of their data than would a major federal monitoring program that has to make contentious environmental decisions based on data collected by volunteer groups scattered around the country. Scientific credibility, legal liability, fear of regulatory reprisals, and defensibility in court litigation can also be factors for deciding how much emphasis to put on data quality. Federal agencies are mandated by law to meet certain quality requirements for data that are shared with the public, and most agencies have their own guidelines and quality assurance plans. Examples can be found in the References. Data credibility plan—The process of assuring others that data are trustworthy can be divided into three components that all improve credibility: quality assurance, quality control and quality assessment (discussed in the following sections). Data credibility can be defined as everything that is done to convince others that the collection of data was done in a thoughtful, systematic, unbiased and careful manner. Documenting a detailed monitoring project plan through use of the workbook forms (Appendix 3) assures the reviewer that all aspects of the monitoring project were given due consideration. Documenting the plan’s implementation serves to assure reviewers that the monitoring methods that were selected actually addressed project goals, participants communicated well with each other, all parties were motivated to make the project work well, adequate resources were allocated to training, and that data were collected, analyzed, and interpreted in a non-biased and collaborative manner. A well-documented project plan and records detailing its implementation are an important context for the more specific quality assurance plan that focuses on sampling design, sampling protocols, and quality control methods. Quality assurance (QA) plan—Quality assurance plans are commonplace in business and industry as well as in established federal monitoring programs. Many examples exist and elements vary according to the specific context and purpose of the monitoring. A particularly effective way to insure credibility in a participatory monitoring project is third-party verification or auditing. Common elements include: · Documenting a data quality control plan (see below). · Employing experts to design sampling plans and peer-reviewing their recommendations. · Developing standard operation procedures (SOPs) such as documented sampling protocols. · Writing reference manuals for the SOPs. · Designing and conducting training programs for implementing SOPs, including manuals, curricula or teaching aids. · Deciding on procedures for periodically evaluating the efficacy and appropriateness of the SOPs. · Documenting changes in the SOPs and providing training to implement the changes. · Evaluating, documenting, and certifying training accomplishments and learned skills. · Conducting continuous oversight or supervision of data gathering. · Periodic third party or independent field checks of data accuracy. · Developing a process for evaluating the efficacy of quality assurance plans and revising them as necessary (see quality assessment below). · Ensuring that the data are analyzed properly and interpreted objectively (see Analysis module).
· Accuracy – confidence that the measurement reflects the actual value, that is, both nonbiased and precise. · Lack of bias – the measurements are not systematically skewed. · Precision – degree of agreement between repeated measurements of the same sample. · Completeness – sufficient samples are acquired to provide useful information. · Representativeness – extent to which the measurements you take actually reflect the state of the indicator that you wish to monitor. Other factors such as detection limits, instrument sensitivity, or sampling at appropriate scales, places, and times can also be pertinent to quality control depending on what is being measured and how. A variety of methods exist for calibrating measurements and preventing data errors. Standard reference materials can be used to calibrate instruments. If field measurements seem to be interpreted differently by each crew member, then field personnel should compare their methods until everyone agrees upon useful criteria to reduce variability or bias in their measurements. Those criteria should then be documented in the sampling protocols and uniformly applied. Data errors can be reduced by double entry, checking for reasonableness, and by having team members review the data for correctness and completeness as it is entered in the field. Quality checks can be conducted by other teams that re-sample the same plots. Regardless of the actual techniques and processes for confirming data validity, methods can always be improved; therefore convincing outsiders that project participants are sincerely committed to data quality also involves periodic evaluation and review of the quality assurance plan. This is sometimes called a Quality Assessment Plan. Quality assessment plan—Are the selected indicators the most appropriate monitoring targets to meet the goals of the project? Do the collected data reflect meaningful changes in the status of the indicators that are being monitored? What components of the Quality Assurance Plan merit improvement? Is the sampling design achieving monitoring objectives in the most effective or efficient manner? Are sampling protocols appropriate, sufficiently documented, adequately taught, and consistently applied? Do users of the data have any concerns about its credibility and how could those concerns be addressed? These and related questions are all relevant concerns for assuring end users that sufficient attention is being paid to the issue of data quality; thus developing a written plan to periodically review these topics also demonstrates foresight and commitment.
|
|