Has Risk Assessment Become Too "Conservative"?

Quantitative risk assessment (QRA) is a science and an industry, and "risk numbers" are both its language and its currency. These numerical predictions of how many persons will suffer disease or death because of environmental exposure, or of the probability that an average person or a particular individual will succumb, now lie at the heart of environmental health regulation, particularly when it involves carcinogenic substances.

The recent controversy over daminozide (Alar) in apples, for example, centered around estimates generated by the Natural Resources Defense Council (NRDC) that as many as 5,300 of the current group of 22 million preschool children in the United States may contract cancer during their lifetime as a result of childhood exposure to Alar. This represents an estimated increase of 1 chance in 4,200 (above the background probability we all have of getting cancer) that Alar will cause cancer in a typical child. The NRDC also estimates that about 5 percent of preschool children ingest substantially more food containing Alar than the aver-age child, and that these children face excess cancer risks approaching 1 in 1,000.

Experts and laypeople alike tend to ask two very different kinds of questions when confronted with numbers like these. One set of questions involves ethical judgments about the acceptability of the stated risks; the debate over whether a risk of (say) 1 in 4,200 is too high will depend on personal and group judgments. These judgments concern the voluntariness of the risk, the magnitude of the probability (perhaps in relation to other environmental, occupational, or lifestyle risks we are more familiar with), the costs of eliminating or reducing the risk, and the real or perceived benefits of the risky product or activity. This acceptable-risk issue pits those who argue that no involuntary risk is acceptable if it can readily be reduced further against those who believe our society has become preoccupied with trivially small dangers. This is a vigorous debate, with divergent views expressed both within the expert community and the general public as well as between these two groups.

The other set of questions has to do with the believability of the estimates themselves. In contrast to the controversy over acceptable risk, the debate over whether risk numbers are credible has begun to resolve itself, at least among practitioners and expert observers of QRA. The general reader may be surprised that this group tentatively has concluded that risk numbers generally are not credible. The conventional wisdom of the experts is that these numbers are systematically skewed in the direction of overestimating risk, because the process used is in danger of being so "conservative"—so overly cautious—as to be a caricature of itself.

The intellectual and regulatory momentum is clearly on the side of the "revisionist" position, which seeks to replace conservative procedures because the status quo is allegedly causing alarmist and counterproductive reactions. The lack of resistance to some of these changes reflects the compelling evidence supporting some revisions, the fact that the public may not be aware that subtle but accelerating changes are under way in QRA, and perhaps simply the natural swing of the pendulum in such matters. In my view, however, the rush to eschew conservatism is fueled in part by an uncritical acceptance of a set of flawed assumptions about QRA, so the pendulum swing may itself be counterproductive. I wish to offer a note of caution against hasty or piecemeal changes, and to suggest a new approach that may preserve the virtues of both good science and prudence.

The case against conservatism

The fundamental logical flaw of conservatism is that it can compromise our ability to make clear choices and set rational priorities. The strongest critics of conservatism view this distortion in the broadest possible terms; conservatism, they say, artificially inflates the relative importance of all proposed measures to reduce health and environmental risks. Some revisionists simply do not believe that the hazards of industrial pollution are as dire as the standard QRA procedures imply. But arguments that focus on the need to reduce existing risk numbers and redress the balance between risk and cost probably exacerbate the tension between the experts and the public, and may backfire. After all, a "realistic" toll of 530 extra deaths from Alar (if revision caused a lowering of this risk number by a factor of 10) might be no more acceptable to the public than a cautious estimate of 5,300 fatalities.

Therefore, a more reasoned and perhaps ultimately more successful argument against conservatism is that it creates imperceptible distortions among different risks, which we cannot redress simply by paying less attention to cancer risk reduction (or by agreeing that we are spending about the right amount even though we have exaggerated the size of the risks). The insidious aspect of consistently analyzing the "worst case" is that some cases are simply "worse" than others, in the sense of being less plausible or less likely to occur. For instance, one typical conservative shortcut is to assume that the most highly exposed individual near a chemical plant or a hazardous waste site lives at the property boundary, and that he or she is downwind of the pollutant source 24 hours a day. In some cases, the resulting risk estimate will be quite conservative, if no one actually lives near the boundary or in the direction of the prevailing winds. In other instances, the estimate may be nearly correct. If the cancer risk estimate cited for the former situation was 1 in 10,000, and the estimate for the latter was 1 in 100,000, the former would seem more risky even though (unknown to the investigator) this estimate was less credible than its counterpart.

Perhaps influenced by these arguments against conservatism, the U.S. Environmental Protection Agency (EPA) has recently begun to reconsider some of the official risk estimates it developed in earlier years. To date, all of the proposed reevaluations have resulted in lowered risk numbers, generally by about a factor of 10. The most noteworthy of these cases involve methylene chloride (a solvent used, among other things, to decaffeinate coffee), arsenic, and TCDD, also known as dioxin.

Potentially more far reaching than these ad hoc changes in specific risk assessments is EPA's September 1988 decision to rewrite its influential series of guidelines for quantitative risk assessment, which had been published in 1986. These guidelines determine which assumptions should be used under various circumstances, and indicate in general terms when professional judgment should supplant formulaic procedures. Although it is too early to tell specifically how the new guidelines will reflect what has been called the new era of post-conservative risk assessments, they may encourage the use of alternatives (see table 1).

Conservatism in perspective

A number of pervasive misperceptions about conservatism cloud the issue of whether risk numbers are credible and QRA procedures are reasonable. The following points refute three of the broad categories of misperceptions.

Existing procedures are not so unscientific or unreasonable

Critics tend to malign different kinds of conservative assumptions with the same broad brush, failing to distinguish those that are gratuitous from those dictated by prudence or common sense. For instance, in contrast to the use of simplistic worst-case assumptions about exposure that could readily be refuted by reliable data, the commonly criticized use of the upper confidence limit when fitting a dose-response curve to animal data is a cautionary step of a quite different variety. This procedure recognizes that as we learn more about cancer potency, the truth may well fail to converge toward a lower result. To put it another way, suppose the owner of a baseball team approached one of his star players four days into the season and asked him to take a pay cut on the grounds that he was batting .050 at the time. The player would doubtless argue that he has always had about a 1 in 3 chance of getting a hit each time at bat, and that his current 1-for-20 string is too scanty a basis for claiming that that underlying probability has changed at all. By the same token, observing 5 tumors in a group of 50 rats does imply that each rat had about a 1 in 10 chance of getting cancer at that dose, but is only weak evidence against the more prudent assumption that the probability might be several times larger.

In addition, it is easy to carp about possible errors of commission in the QRA process without acknowledging that various errors of omission may make risk estimates more "nonconservative" for all or part of the human population. Of most significance, risk assessments commonly fail to account for the often-dominant indirect exposures (such as inhaling organic compounds that volatilize from hot tap water during showering and bathing) and for the likelihood that individual humans differ widely in their inherent susceptibility to carcinogenic stimuli (we currently assume that all humans are as homogeneous in their responses as are the inbred strains of rodents we test in controlled environments). Thus, the current mix of assumptions may contain certain margins of safety necessary to account for our inability to fully flesh out important considerations.

Beyond that, the common characterization of QRA as a "cascade" of conservative steps that yields progressively more unbelievable estimates may confuse issues of probability and magnitude. It is true that if one multiplies five estimates that each have only a 5 percent probability of being underestimates, the product will have much less than a 5 percent chance of being too low. However, many of the individual uncertainties in risk analysis are right-skewed; that is, the highest possible values in the "tail" are much greater in absolute terms than the more central values. The fact that extreme values are unlikely to occur becomes less and less important as the consequences of those values being true become greater. For example, the average indoor radon level in a sample of 5,000 homes in Pennsylvania was about 10 picocuries per liter (pCi/I) even though a randomly selected house had only about a 20 percent chance of containing more than 10 pCi/l. Decision makers and the public need to consider that while it is easy to ridicule a risk estimate for being exaggerated (in the sense of unlikely to be too low), such estimates may be more reasonable than less cautious ones.

Data do exist to validate some existing numbers and procedures

Critics of conservatism sometimes fail to acknowledge that evidence exists to support the "reality content" of risk assessment procedures or of the risk numbers themselves. For example, researchers at the Harvard School of Public Health recently concluded that on average, the linear dose-response function is not unduly conservative; for many chemicals, the best-fitting curve was in fact steeper at low doses than at higher ones. Similar challenges to the notion that the current estimates are systematically conservative come from recent studies of the dispersion models used to predict the movement of pollutants in air and water, which have shown that the models often underpredict actual concentrations, especially when the terrain or atmospheric environment is complicated.

The most direct "reality check" on QRA involves comparing the predictions of animal extrapolation to the actual cancer toll among humans exposed to known levels of a particular substance. Such a comparison can only be made for about two dozen substances (for example, cigarette smoke, vinyl chloride, and chromium) where both human and animal data on exposures and tumors are reasonably reliable. The basis for generalization is therefore limited, and the human potency estimates may be nonconservative (they generally come from data on small groups of relatively healthy workers). However, one research group recently found that, on average, conservative extrapolation procedures yield estimates of human cancer potency that agree fairly well with the actual potencies observed in epidemiologic studies.

Alternative methods may substitute one set of laws for another

The prospect of replacing conservative assumptions with "best estimates" of actual risk may be no less problematic than the status quo. Although conservative estimates have been widely derided as "policy choices masquerading as scientific facts," central or average estimates themselves embody subtle value judgments regarding the implicit social costs of erring on the high or low sides. In this respect, best estimates are no better than conservative ones, which simply strike this balance more in favor of caution about underestimation, and may reflect a desire to minimize large absolute errors of underestimation. In addition, while it is desirable to reduce the ambiguity about how conservative estimates of different risks are, one can show that errors in ranking uncertain risks are also endemic even when best estimates are consistently used.

Reframing the question

Many of the problems engendered by the use of conservative risk numbers (as well as their "real" counterparts) can be overcome by one deceptively simple step—abandoning the quest for single estimates of risk in favor of quantitative descriptions of the uncertainty surrounding these numbers. Such descriptions, which would take into account random and systematic sources of uncertainty in potency, exposure, and uptake, would reveal all of the possible true values of risk and the likelihood associated with each.

If uncertainty analyses became routine, we could move beyond the narrow debate over whether the estimates were too high or too low and could instead choose the degree of conservatism explicitly and with appreciation of the scientific nuances and societal value judgments specific to each case. For example, researchers from the National Institute of Environmental Health Sciences recently conducted an uncertainty analysis showing that if the EPA wanted to retain an estimate of methylene chloride potency that was a 95th-percentile conservative estimate, it might well have raised the official estimate by a factor of 1.5 (rather than lowering it by a factor of 9, as was done).

Quantitative uncertainty analyses can also facilitate dialogue between risk managers and the public concerning how much society is willing to pay to reduce the possibility of particular levels of harm, and can help regulators perceive which uncertainties are dominant and thereby set strategies for research. All of these benefits come at a price, however. Uncertainty analyses are expensive to conduct, sometimes difficult to explain, amenable to subtle manipulation by interested parties, and may be foreboding in that they reveal how little the experts actually know about the likelihood of different levels of harm. Nevertheless, the real challenge of QRA in the next decade will be to recognize that while acknowledging uncertainty may be as difficult as stepping out of one's own shadow, only through the attempt can we discern from what direction the shadows are cast and in which directions to move so that they might ebb.