Contact FOTP: OrdToxics@aol.com
Comments from Review of
Former Fort ORD OECert Analysis Draft Report
Prepared by Julie Naster, Frey-Naster Associates, LLC
Introduction
On behalf of Ecological Consultants for the Public Interest (ECPI) for the Fort Ord Toxics Project (FOTP), a review of the Former Fort ORD OECert Analysis Draft Report (OECert)(QuantiTech, Inc., 1996) was performed to evaluate the OECert itself as well as its use as a human health risk assessment tool for the Engineering Evaluation/Cost Analysis - Phase I (EE/CA) for Fort Ord.
The general purpose of the OECert was to provide cost and human health risk data for alternatives for removal actions to be evaluated in the EE/CA. Thus, the validity of the model used to develop the OECert output as well as the meaningful use of the output in evaluating risk from residual ordnance and explosives (OE) in the EE/CA were considered in this review.
The scope and input data for the EE/CA were developed on the basis of historical studies and data collected in connection with previous removal actions. While the data and results from the studies are of some concern with respect to the final results of the risk evaluation, they are not the subject of this review. Rather, this review focuses on the validity of the methodologies used in the risk evaluation process. The EE/CA and the use of sound methodology for its development are important because the results will be used to determine levels of cleanup at the sites covered in the document and will also set precedent for removal actions that will be addressed by the Phase II EE/CA at Fort Ord.
Summary of Findings
Findings from this review are summarized under two categories. First, concerns with the OECert risk model and its implementation are discussed, followed by concerns with use of the results in evaluating alternatives in the EE/CA.
OECert Risk Model Concerns
The OECert Risk Model was developed under contract to the U.S. Army Corps of Engineers by QuantiTech (U.S. Army Engineer Division, 1995). The purpose of this model is to serve as a tool for estimating public and occupational exposures to OE after various removal action scenarios and the cost of implementing those scenarios. The model has been used in the past for other OE sites. Because safety is the primary concern of the FOTP, this review addresses the validity of the risk portion of the model without addressing that of the cost portion.
The model utilized existing data from past removal actions to estimate the areal (based on surface area rather than volume) density of OE in previously defined sectors at Fort Ord. Existing sampling data from current investigations were used to develop a representative depth profile for OE, which did not vary among the sectors. A Poisson distribution process (a commonly used statistical tool) was then used to establish the probability of an individual encountering OE on a single visit, given the density of OE and size of the area visited. Use projections were applied to the individual probability result to determine the number of exposures likely to occur on an annual basis. Exposure was defined as a participant (or site visitor) being in proximity of ordnance with or without knowledge of it. Risk of actual injury or fatality from an exposure was not considered in the model.
The following general comments are made with respect to the OECert model:
- The key input parameters for the model are OE density at each depth category, the area traversed by each participant in a given activity, and the number of participants performing a given activity on an annual basis. The source of the density data is documented in the OECert. The number used in the Appendix A calculation for the area traversed appears to be conservative. However, no basis for the number of participant visits is provided. This number will vary hugely based on the population and use assumptions and estimates that are used. Even in a remote area, the number used in the example (63 cyclist visits) seems very low (i.e., not conservative). Use data are key to the outcome of this model. Without clear documentation as to what data were used and how they were developed, the validity of all model results is suspect.
- Documentation for the model is inadequate. There is no reference list. Without indication of quality assurance procedures and peer review used to validate the model, the model is not fully credible.
The following specific comments and concerns were identified with respect to the OECert model:
- The executive summary states (page 3) that, "only a subset of the overall EE/CA Phase 1 sites were subjected to an OECert analysis." No further explanation is given as to which sites were analyzed and why. This leaves the reader wondering why others were not analyzed and whether failure to analyze them might be significant.
- Residual density estimates were calculated using depth dependent sweep efficiencies (page 4). The only documentation for these efficiencies in Table C-1 is AESCH guidance. No discussion of the applicability of the guidance nor a full reference is given, leaving the reader to wonder if the correct sweep efficiencies were used and how use of different efficiencies might affect the results of the risk estimate. Additionally, sweep efficiencies are not defined in the document, which adds to the difficult a reader has in interpreting their contribution to the analysis.
- An exposure is defined as a participant in an activity being in the proximity of ordnance, with or without knowledge to the presence of ordnance. There is no definition of proximity. Does it mean within a 10-foot radius or a 1-foot radius or touching the ordnance? Because model results are used comparatively and the actual risk to human health from an exposure is not addressed by the model, this lack of definition does not have an opportunity to affect the outcome in the EE/CA but still detracts from the credibility of the model. (See the general comments below on the EE/CA for more discussion of the meaning of the risk numbers.)
- The site comparison (Table ES-8) to other Formerly Used Defense Sites does not seem relevant. Without some measure of absolute risk or at least some historical data of human health effects experienced on other sites, a comparative ranking is useless. Just because the Fort Ord site places among the low to moderate exposure risk sites, does not mean that the risk or future risk there, given proposed land uses, is acceptable. In fact, there is no way to determine the acceptability of the risk. (Again, see the general EE/CA comments for more discussion of the meaning of the risk numbers.)
- Use of 0% for the distribution of OE from a depth of 4-10 feet is misleading (page 13). Because no investigations were made at this depth, the depth should not receive a numeric value in the depth profile nor should it be included in the quantitative analysis, unless assumptions for its use are made and demonstrated to be conservative.
- There is an error at the top of page 14. The first line should read 1-2 feet instead of 0-2 feet.
- Activities representing those described and documented in the Fort Ord Base Reuse Plan from December 13, 1995 were used to develop specific site level activities (page 14). This source appears to be different than the one cited in the EE/CA (page 2-6), which is a December 1994 document and also different than the source cited in Table C-1, which is called the Fort Ord Base Disposal Plan. Are these all the same document but sloppily referenced? If they are different documents, it must be shown that the risk analysis was performed based on the most conservative use shown in any of the documents. Assumptions and resulting data regarding future use are key variables affecting the risk estimates. Therefore, use of the most recent land use plan is important throughout both the EE/CA and the OECert model.
- Relatively detailed description of the cost model is included in the body of the report, while a less detailed sample calculation showing the risk model is relegated to an appendix. This appears to be indicative of the emphasis placed on cost rather than human health and safety risk in the OECert model.
- The effective area is defined as the minimum of the sector area and the area that an individual covers while mountain biking (page A-3). The larger the effective area, the higher the probability of exposure. Therefore, the maximum of the sector area and the area that an individual covers should be used to produce conservative results.
EE/CA Risk Concerns
Risk results derived from the OECert were used as the sole measure of human health risk in evaluation of removal action alternatives in the EE/CA. The following concerns address the way in which these numbers were used in the EE/CA decision process:
- Injury, permanent disability, and death are the possible outcome of an explosive incident involving OE. OE hazard risk is the chance (probable risk) that the OE encountered will cause a hazard. In simpler terms, the OECert risk is the probability of encountering a UXO and the hazard risk is the probability the UXO encountered will cause an injury (page 2-99). This statement reflects the inadequacy of OECert risk, by itself, to give a meaningful indication of actual risk of injury or fatality to members of the public. The hazard risk, as described, is a key link to an absolute evaluation of risk. The EE/CA only addresses risk in a comparative manner. Thus, the alternatives can only be ranked in relative terms. Without absolute risk measurements and proposed or agreed risk acceptance criteria, it is not possible to evaluate the acceptability of residual risk following removal action. Likewise, without an understanding of absolute risk, a very conservative approach should be used in making and implementing decisions on future land use.
- It appears from the exposure predictions under the various alternatives analyzed that risk of encounter can only be reduced so far. What are the limitations to further risk reduction? Is the limitation derived from limitations with the detection technology? A discussion of this as it pertains to risk reduction would be helpful. Available technologies are described in Chapter 4.0, but without any effectiveness data. It appears that the effectiveness of detection technology was not factored into the technical feasibility for each alternative.
- The results of health and safety evaluations are lost in the comparison method used. With an equal weighting for all criteria, the importance of health and safety is diminished. A weighting scheme that reflects the values of stakeholders is key to producing a meaningful result from the EE/CA. Dialogue and negotiation among stakeholders should be used to develop a more consensus-based weighting system.
References
QuantiTech, 1996. Former Fort Ord OECert Analysis. Technical Report 96R015. Document prepared for EARTH TECH, Contract Number: DACA87-95-0017, by QuantiTech, Inc., Huntsville, Alabama, December.
U.S. Army Engineer Division, Huntsville, Alabama, 1995. Ordnance and Explosives Cost-Effectiveness Risk Tool (OECert). Final Report, Version E. Technical Report 93R004vE. Prepared by QuantiTech, Inc., Contract: DACA87-93-C-0036. August.
The FOTP website is a SEEK MapCruzinTM project.
Created and Maintained by Michael R. Meuser
Copyright © 1997-1999 Fort Ord Toxics Project
All Rights Reserved