Sizable uncertainties are associated with each element of quantitative environmental risk assessments. They arise because physical, chemical, and biological phenomena are often difficult to measure and do not function in straightforward, predictable, and linear ways. Although large, ubiquitous, inevitable, and only partly reducible, these uncertainties need not impede decisions about how to deal with environmental problems. Supportable decisions can be made if uncertainties in risk estimates are clearly presented, are reduced where it is cost-effective to do so, are accounted for in comparisons among environmental risks, and are the context for making conservative assumptions.
Most people, and some policy-makers, do not recognize the extent of the uncertainties involved in risk assessments that form the basis of many decisions about environmental pollution. Decisions about environmental risks must be made in the face of uncertainties that are far beyond the range of people's commonplace experience. When legislators consider information about risks to health and the environment in setting goals for environmental protection, when agency heads evaluate risks in setting priorities among risk reduction programs, and when state and federal regulators weigh risks in setting limits on emissions or in restricting the use of particular products or substances, their judgments are never made in an atmosphere of certainty. The uncertainties they encounter almost always may be reduced by gathering additional information, but time and cost constraints and some fundamental limits will always leave a sizable amount of uncertainty in any risk estimate.
How sizable is "sizable"? A full assessment of how much or how little is known about the quantities of harmful substances released from different sources, the concentrations of these substances in the environment, and the toxicity of these substances would reveal that actual environmental risks could be higher or lower than estimated risks by factors ranging from tens to hundreds or more. If such uncertainty were characteristic of an individual's income over a one-year period, that income could plausibly range between $5,000 and $500,000, or even between $500 and $5 million.
Evidence shows that the uncertainties in environmental risk estimates (quantitative assessments of health or other risks to humans and other organisms in the environment) are large, ubiquitous, inevitable, and only partly reducible. The magnitude of these uncertainties would be less if physical, chemical, and biological phenomena were easy to understand and to measure and if these phenomena functioned in straightforward, predictable, and linear ways. However, the real world operates in complex ways.
First, important physical or biological relationships are often complicated. For example, the concentration of a substance generally increases in a linear way with the average quantity released. However, the relationship between concentration and quantity becomes less predictable and less linear as situations common in the real world intervene. Shifting or variable winds, physical objects such as buildings, and other factors that affect the dispersion of a substance in the environment can make estimates of concentrations relatively uncertain.
Second, phenomena that are easy to measure, such as average concentrations, can sometimes lead to over- or under-estimation of other phenomena that are not so easy to measure, such as instantaneous or peak concentrations. For example, wind-tunnel experiments demonstrate that measurements of the average concentrations of propane within a propane gas cloud can lead to under-estimation of the cloud's flammable area because flammability is affected both by average and peak concentrations.
Third, the links between the release of a substance and the consequences of that release are nonlinear, involve several complicating factors, and are probabilistic. For example, records of approximately 500 accidental releases of chlorine gas indicate that the likelihood of human injuries increases with the quantity released, but not proportionately; and the number of persons injured increases, but only roughly in proportion with the quantity released. Moreover, quantity is only one of many factors that determine the ultimate effects of a chemical's release.
Steps in risk estimation
Uncertainty pervades each element of environmental risk assessments, as an examination of estimations of the risks to humans from the release of a harmful substance into the environment illustrates. These estimations all deal (explicitly or implicitly) with the following questions: What is the probability that release of a substance will occur? What quantity of this substance will be released? How will the concentration of this substance change as it disperses from the point of release? How many people or other organisms in the environment will be exposed to this substance? How much of the delivered dose of this substance will be taken up by organisms? And what will be the relationship between a par-ticular dose of this substance and an organism's response—that is, how will the toxic effects of this substance increase with increased dose?
The mathematics in a quantitative risk estimate may become complex, but essentially each of the elements enters into this estimate in a multiplicative way:
Risk = probability x quantity x (1/dispersion) x population x uptake x dose vs. response
An increase in any of these six elements leads to higher estimates of risk levels.
There is always some uncertainty about each of these elements in the risk estimation process. In the discussion below, the terms "extreme certainty" and "extreme uncertainty" denote opposite ends of the spectrum of how well a particular phenomenon is understood, and the modifiers "high," "fair," and "moderate" depict gradations of this understanding (see table, p. 11). These gradations, though arbitrary, are appropriate to the range of uncertainties found in environmental risk analysis.
Range of uncertainty and variability in environmental risk assessments
A gallery of uncertainties
A large body of scientific and engineering studies is available to describe the extent of current uncertainty about each of the elements in the formula above. This information can be summarized as follows.
Probability of a release of a harmful substance
Estimating the probability of a release of a harmful substance may be unnecessary for many environmental risk assessments. It may be known with certainty that a factory is discharging wastes into a waterway or that farmers are using a particular pesticide on a crop. There is no doubt that cars emit pollutants. However, the 1984 release of methyl isocyanate from the Union Carbide plant in Bhopal, India and the more recent spill of oil from the Exxon Valdez off the coast of Alaska were accidental. Estimating the probability or frequency of such accidental releases is the first step in many environmental risk assessments.
There is sizable uncertainty about estimates of the probability of accidental releases of harmful substances. Historical data offer some guide to this probability for frequently occurring accidental releases, but are less instructive about some rare kinds of accidents involving certain substances. Examples of such accidents are a large release of radioactive substances from a nuclear power plant or a large release at a chemical plant. Mathematical models developed for estimating the frequency of accidental releases of radioactive substances at nuclear power plants offer a guide to the probability of accidental releases of harmful substances at facilities, such as chemical manufacturing plants, that carry on activities similar in complexity to those conducted at nuclear power plants. Since the models produce fairly uncertain estimates of the probability of accidental releases of radioactive substances, it would appear that most estimates of the probability of chemical releases would be no better than fairly uncertain.
Quantity of a harmful substance released
Estimates of the quantity of harmful substances released into the environment improve with the amount of information gathered. For example, an analyst could estimate the quantity of pollutants emitted into the air by industrial facilities using knowledge of the facilities' production volume and estimates by the Environmental Protection Agency (EPA) of the number of pounds of each pollutant released in the production of one ton of each product. The resulting estimate would be fairly uncertain but could improve to moderately certain if it were based on additional input from the EPA's Toxic Release Inventory, in which 22,000 large manufacturing facilities report estimates of their annual releases of toxic substances.
Dispersion of a harmful substance and resulting concentrations of that substance in the environment
It is possible to obtain results from mathematical simulations of air and water dispersion that correspond well with the results of a particular test release of a substance into air or water. However, in practical situations, estimates of concentrations at particular points in the dispersion pathway range from fairly to highly uncertain.
When risk assessors have actual measurements of concentrations to work with, they can omit estimation of the probability of a release of a substance, the quantity of the substance released, and the dispersion of that substance. However, actual measurements of concentrations of particular substances in the environment reveal that concentrations are highly variable. For instance, measurements show that ambient concentrations of some pollutants are moderately invariable to moderately variable from day to day in a particular season. These concentrations may vary by a factor of 5 to 10 between warm and cold seasons and among different years. Actual measurements show that radon concentrations among the homes in a given locale may be fairly variable. In addition, they show that human exposure to some chemicals in the workplace may be extremely variable among individual work situations.
Population exposed to release of a harmful substance
The pattern of the population of people living in, working in, and visiting an area near the source of a release of a harmful substance influences the degree of risk of adverse effects from that release and, thus, the degree of risk of adverse effects among individuals. In the United States, some industrial sites may have no residential population within 500 yards, or even 3 miles, while others may have population densities ranging from less than 1 to more than 2,000 persons per square mile. In the absence of site-specific data, any estimate of the population potentially exposed to the release of a harmful substance from such sites would be extremely uncertain.
Uptake of harmful substances by humans and other organisms
In the past decade, there has been intense interest in trying to understand the biological mechanisms at work when a human's exposure to a harmful substance leads to harmful concentrations of that substance in each of the individual's internal organs. Scientists have tried to advance such understanding by developing pharmaco-kinetic models that reflect how the physi-ology of humans differs from that of test animals with respect to uptake, metabo-lism, and excretion of particular chemi-cals. But even the most carefully constructed of these models must rely on estimates of a number of variables, such as how fast a chemical passes from the blood into an organ, that are either diffi-cult or impossible to measure directly in humans and test animals. Pharmacokinetic models may modify risk estimates in significant ways, but their structure and the data they contain make these modifications moderately to extremely uncertain.
Relationship between dose of a harmful substance and adverse toxicological response
A number of variables affect the dose-response relationship—that is how organisms respond to a particular dose of a harmful substance and how this response changes as the dose increases. The toxicity of a particular dose of a substance not only varies across species, but among individuals of the same species. Sex, age, size, diet, and the route of exposure to a substance, among other variables, affect how toxic increased doses of a substance are to organisms.
Most acute toxicity estimates—for example, estimates of the dose of a substance that would be lethal to half of the subjects in a laboratory test—are moderately uncertain. Estimates of most cancer potency values are even more uncertain This is because test animals are given higher doses of substances suspected to be carcinogenic than humans would normally receive, and because epidemiologic data linking the incidence of human cancer with such substances is often unavailable or inconclusive. Thus cancer potency estimates are fairly to extremely uncertain.
Measurement error
Measurement error increases the uncertainty associated with each of the above elements in an environmental risk assessment. Given clearly specified protocols and well-calibrated equipment, measurements of physical quantities—such as the concentrations of substances in air, water, food and the tissues of organisms—may be highly certain or even extremely certain. But practical constraints, including lack of training and supervision in the use of measurement devices and variability in the precision of these devices, may limit the accuracy of some measurements to the moderately certain range.
Of all the elements of a risk assessment, measurement error generally introduces the least uncertainty. Since the range of uncertainty for dose-response relationships often extends to the extremely uncertain, these relationships are generally the most uncertain element of a risk assessment, followed, in rough order of increasing certainty, by uptake by organisms of a harmful substance, population exposed to the release of a hazardous substance, probability of a release of a hazardous substance, dispersion of a hazardous substance, and quantity of a hazardous substance released (see figure, p. 13). However, this ordering varies with the circumstances in which a particular risk assessment is made. For example, if a wealth of definitive toxicity data are available but other phenomena, such as dispersion patterns, are poorly understood, estimates of dose-response relationships may not be the most uncertain element of the risk assessment.
Overall uncertainty
Uncertainty in each of the elements of a quantitative risk assessment combines to produce an overall uncertainty in the estimate of a particular risk. To date, only a few studies have carefully integrated the uncertainty associated with individual elements of a risk estimation into an overall estimate of a risk and its overall uncertainty. In these studies, estimates of human cancer risks due to exposure to chemicals have been shown to be moderately to extremely uncertain.
Typical ranges of uncertainty in the elements of some environmental risk assessments
Policy implications
Uncertainties in environmental risk assessments are present in all elements of the assessment process, are large, and are inevitable because of practical limitations on the ability to perform enough precise measurements to capture the sizable amount of natural variation and randomness in physical and biological systems. These uncertainties make decisions about environmental problems difficult, but there are several approaches to deal with them.
In order to make supportable, robust, and sound decisions, risk assessors should present the particular level of uncertainty in the risks they are estimating. Risk managers should reduce uncertainty when the reductions would be cost-effective and would affect the choice of risk management option. They should make comparisons among risks and comparisons among actions to manage risks with uncertainties in mind. And, if they must apply conservative assumptions, they should do so in the context of the particular uncertainties of the risk being addressed.
Current environmental risk assessments rarely present estimates of uncertainties in a clear and consistent manner. Competent engineering, toxicological, and other scientific studies routinely indicate the precision of the measurements they use and the confidence with which they estimate relationships, but they present this information in a variety of ways. The Environmental Protection Agency could exercise leadership in this regard by developing standards for the presentation of its own quantitative information so that variability and uncertainty would be prominently and unambiguously displayed. Ideally, EPA would present this information in both numerical and graphic ways, would use graphic displays and statistical distributions that reflect the wide variation routinely found in environmental data, would seek a definitive system of terms and measurements of variability, and would adopt consistent approaches to some recurring problems in the presentation of data. Risk analysts confront these problems when they have to make sense of concentration measurements from several different studies. To do so, they might need to combine some point estimates, some measurements presented in high-low ranges, and some observations that were below the detection limits of measurement devices.
Concerning the reduction of uncertainties encountered in risk estimations, risk managers should be selective. Reducing the uncertainty in any one or more of the elements of a risk assessment will reduce the overall uncertainty of a risk estimate, but not all such reductions are equally cost-effective. In a particular risk assessment, for example, it may be possible to improve estimates of the quantity of a particular substance released from a particular kind of facility from moderately uncertain to fairly certain. However, improving estimates of the population potentially exposed to such a release from extremely uncertain to moderately uncertain would reduce the overall uncertainty more and might also cost much less to accomplish. Risk managers can use the economic concept of the efficient frontier to determine which reductions in uncertainty are likely to reduce overall uncertainty the most and at the lowest cost.
Risk managers should not consider cost-effectiveness to be the only criteria for making an investment in a reduction of uncertainty. Such an investment is worthwhile only if the reduction would be cost-effective and would affect the choice of risk management option.
With respect to making comparisons among environmental risks and among potential actions to manage them, risk managers should consider how the uncertainty of the risks in question increases or diminishes the significance of these comparisons. For instance, it may be appropriate to consider the magnitude of risk A and of risk B in deciding which risk should be managed first, but a comparison of these magnitudes may be misleading when estimates of the magnitudes are highly uncertain. Similarly, it may be appropriate to consider costs and benefits in deciding whether to take action X or action Y to manage a particular risk, but a comparison of the average costs with the average benefits of these actions can be misleading when the uncertainties associated with the risk are large.
To help end debate over the application of conservative assumptions, uncertainties in each element of a risk assessment must be explicitly acknowledged and approximated.
To make more meaningful comparisons and better decisions, risk managers can make use of conceptual tools from the field of decision analysis. For example, Monte Carlo simulations take advantage of the ability of desktop computers to quickly calculate a risk thousands of times. By randomly "drawing" a value for each uncertain variable (using estimates of the uncertainty of each variable to determine how likely each draw should be), these simulations construct a distribution of the overall uncertainty of the risk out of repeated combinations of the random draws from each element in the risk formula.
With regard to making conservative assumptions in the context of uncertainty, risk managers should consider when and how to apply these assumptions. The timing and appropriateness of their application have been a central argument in environmental risk assessment for twenty years. Some critics of environmental regulation, as currently formulated, assert that conservative assumptions have resulted in grossly inflated risk estimates and, thus, in regulations that are too stringent. Others assert that these assumptions are needed in one or two of the elements of environmental risk assessments, most often in estimates of dose-response relationships and the population potentially exposed to a risk, in order to compensate for various uncertainties in other elements of the assessments or for elements that may have been omitted in the assessments.
Three actions are required to end this fruitless debate. They all involve delaying the application of conservative assumptions until the overall risk is estimated and the uncertainty of the risk estimate is assessed, because preemptive safety factors obscure the extent of what is known and what is not about the risk in question. The first action is to explicitly acknowledge and to approximate the uncertainties in each element of the risk estimation process. The second is to carry the uncertainties through the calculation process and to calculate not only the median or mean risk, but its overall uncertainty as well. In its recently released "Guidance on Risk Characterization for Risk Managers and Risk Assessors," EPA appears to endorse these actions in whole or in part.
The third action to end the debate over conservative assumptions is to develop and apply decision criteria that explicitly address acceptable risk levels for individuals and the overall population. If risk managers decide that public policy requires the extra protection that conservative assumptions provide, these assumptions should be reflected in the criteria on which risk management decisions are based and not enmeshed in the risk assessment, where they are hidden from view. Moreover, these criteria must reflect the uncertainties inherent in risk estimations. Risk managers may judge it publicly acceptable for a maximally exposed individual to have a risk of 1 in 10,000 for suffering serious adverse effects from a harmful substance if the risk estimate is at least moderately certain. If, however, the risk estimate is highly or extremely uncertain, the risk might have to be lower—say 1 in 100,000—to be considered acceptable. Similarly, risk managers may judge a risk management policy satisfactory if there is a 75 percent chance that its benefits exceed its costs, but if there is a greater than 25 percent chance that its costs may actually exceed its benefits, they may seek additional information to determine whether a different policy would make a better choice for managing the risk in question. It should be noted that the criteria used in these examples are illustrative. At EPA and other government agencies, criteria for acceptable risk are still evolving.
It is the job of government risk managers to represent the views and interests of the public in setting and applying these criteria. Risk assessors have the obligation and the ability to provide what the risk managers need to do that job—that is, a description of the nature and the magnitude of risks, including the uncertainty of these risks.
Frederick W. Talcott has recently been a visiting scholar in the Center for Risk Management at RFF. He is an operations research analyst in the Office of Policy Analysis at EPA. The views in this article do not necessarily represent those of EPA.
A version of this article appeared in print in the May 1992 issue of Resources magazine.
A version of this article appeared in print in the May 1992 issue of Resources magazine.
