How Much is Too Much? Thoughts About the Use of Risk Assessment for Countries in Transition and the Developing World

The success of risk assessment as a tool for environmental decisionmaking in the United States has encouraged experts to recommend it for use in countries in transition and the developing world. With the end of the Cold War, the countries of Eastern Europe and Central Asia began to ask for help in solving their centuries of accumulated problems. It was hardly surprising that Americans would proffer their best tools. For example, the World Bank and the U.S. Agency for International Development use risk assessment and comparative risk assessment when setting priorities. Local experts are increasingly using these tools as well.

The magnitude of environmental problems facing most of these countries, as well as the persistent weakness of their environmental institutions, leads us to question whether countries still at the level of very basic environmental policy choices are well served when they are encouraged to undertake sophisticated risk assessment procedures. Formal risk assessment, as practiced in the United States, features written descriptions of the risk-creating situation being analyzed, the assumptions and methods of analysis used to reach conclusions, any uncertainties regarding those conclusions, and recommendations for action based on the existing legal and regulatory structure.

Based on a short review of why and when risk assessment is used in Western countries, as well as the environmental conditions in the countries in transition, we ask here whether full-blown risk assessment is the best way for countries with weak institutions and very basic problems to identify their priorities. We believe that sophisticated risk assessments are not useful in such contexts. They often recommend the wrong activities from a practical point of view, and often yield irrelevant results because they ignore institutional issues. In this sense, they represent a misplaced use of scarce resources.

Much cruder methods will often do. If risk assessment is to be used, we believe an important part of the analysis should include a careful consideration of the capacity, ability, and will of these countries to accept the recommendations that derive from such analysis. Analyzing the strength and competence of a country’s regulatory institutions is not normally a component of risk assessment practice in countries with weak regulatory regimes, but we think it should be.

Evolution of Risk Assessment in the United States Over the Past 50 Years

Risk assessment as it is practiced in the United States is closely connected with the existence of institutions and a level of economic development not commonly found elsewhere in the world. The art and science of policy analysis, of which economic and risk analyses are a part, have evolved over the last half-century to serve the kinds of decisions American circumstances require. Most importantly, these conditions include working systems of laws and effective and functioning regulatory agencies.

With these institutions in place, the U.S. government began in the middle of the 20th century to address an environment degraded by many years of industrial activity, careless municipal waste disposal, and uncontrolled vehicle emissions. It soon became clear that some problems were easy to diagnose and solve, but others were more difficult. For example, the problem of a Potomac River made unswimmable by raw sewage was not a hard one to solve, in principle, at least. Sewers had to be built, connected to each other, and their contents treated before the water was released back into the Potomac. Once the political will was found and money was appropriated, the technical problems associated with designing, building, and operating the treatment system were ones that engineers could happily solve. Solving this problem did not take a complex risk analysis, or a sophisticated cost-effectiveness analysis.

A more difficult problem emerged in the early 1970s: the presence of very small amounts of a possible cancer-causing agent in the U.S. beef supply. “DES,” the culprit, was a growth-enhancing hormone analogue used as a feed additive. The U.S. Food and Drug Administration (FDA) could not ban use of DES without eliminating beef from American diets for years. Nor could the agency ignore its presence. According to the Delaney Clause, a 1958 food safety provision that set a zero-risk cancer standard in foods for humans, there must be “no residue” of any carcinogenic substance in meat, among other provisions. And so FDA invented a new decision method, based on the theory that the risk posed by some infinitesimally tiny amount of DES could be negligible.

By the mid-1980s, efforts were under way to address basic environmental issues. The remaining problems were more subtle and difficult, their risks less clear, their costs apparently higher, and the benefits of investing in them not necessarily obvious. Risk analysis, adapted from FDA’s approach to DES, has helped environmental policymakers judge the utility of different solutions to these more complex problems.

The use of chlorine gas to disinfect drinking water provides a good example. Chlorine gas is cost-effective and has a key advantage over other methods: a microbicidal residue remains in the water hours after treatment, suppressing microbial growth in long distribution pipes. However, using chlorine gas also presents risks: chlorine stored in quantity can be very dangerous, and chlorinating raw water that contains large amounts of organic matter will generate chloroform and other chlorinated by-products that may cause cancer. Water utilities found that by filtering raw water before chlorinating it, they could greatly reduce, although not wholly eliminate, formation of by-products. Some risk from exposure to these by-products therefore remains. But over the past few years, the U.S. Environmental Protection Agency has used a sophisticated risk analysis to conclude that, at present levels, these by-products pose a negligible risk, and so use of chlorine for water treatment should continue.

In addition to risk assessment that asks how many illnesses might occur as a result of exposure to some substance, risk assessment also has been used to compare different kinds of risks in order to set regulatory or budget priorities. A significant recent change in the use of risk-based tools in the United States is the inclusion of affected publics in the risk-assessment process. This shift represents a response to public criticism and an evolution from a scientific exercise to one that more broadly addresses the needs of both policymakers and the public.

From one side, community-based activists lambasted risk assessment for not generating useful information and being technically indecipherable. And from the other side, those who bore most of the costs of cleaning air, water, and land blasted it as rigid and biased. This discontent found expression first in vocal criticisms of the Superfund program and U.S. Department of Energy practices. It was given intellectual respectability by a series of reports published in the mid-1990s by several National Research Council (NRC) committees and the Commission on Risk Assessment and Risk Management, which was established under the 1990 Clean Air Act Amendments. These committees essentially called for modifying the FDA-based way of doing complex risk assessment to address these issues and increase participation by ensuring greater inclusiveness and openness with respect to the preparation of the analysis and its use.

In the United States today, risk assessment is increasingly regarded as a means for translating the best available scientific information about risk into language that assists open and public processes for environmental decisionmaking. In democracies, public consent is recognized as a necessary element in setting effective environmental policy. Moreover, experts recognize that many of the judgments made within the risk assessment exercise reflect subjective decisions closely related to public values.

Marked Differences in Environmental Conditions and Institutional Contexts

Environmental conditions in many countries in transition and in the developing world are markedly different from those in the United States. Heavy emphasis on industrialization in the former Soviet bloc left extensive damage, which has been documented at length by the World Bank, the U.S. government, and numerous independent experts. Russian environmental leader Alexey Yablakov has described his own country in this way: “if we compare the planet with a communal apartment, we occupy the dirtiest room.” According to observers such as the U.S. Department of State, a harmful by-product of China’s rapid industrial development in recent years has been increased air and water pollution, which will be a serious problem in China for years to come.

The institutional context for environmental policymaking in countries in transition and the developing world also differs from the United States. Although environmental laws are on the books, most of these countries have not yet put basic environmental controls in place. In some, the use of law to address such problems is not a tradition. In others, enforcement is weak or nonexistent, regulatory systems are frail at best, or the level of government commitment is uncertain.Russian critics have been very vocal about their disappointment in the performance of the Russian State Committee for Environmental Protection. Yablakov has said, “One gets the impression that [the committee] is just treading water. It adopts a whole load of good programmes on lead, on dioxins—but doesn’t carry them through. Not even a quarter of them come to fruition, less than a tenth do.”1 Now even this weak institution has been closed down. In May, President Putin of Russia signed a decree abolishing the State Committee, which had already been downgraded from a ministry several years before by President Yeltsin.

Are these the right conditions for the use of complex risk assessment or other environmental priority-setting tools? Tools to inform policy decisions presume an ability to use these to carry those decisions out and a willingness to act. Risk assessment in the United States primarily is used to guide policy, and the entire process is predicated on “rule of law” with institutions that have adequate expertise and power to carry through the recommendations that result. Without this ability to follow through, we worry that risk assessment will be purely a paper exercise that will divert much-needed resources.

Countries such as Russia and China certainly do not lack people with the skills to do sophisticated risk assessments. On the contrary, scientists with world-class skills and training are found in many parts of the transitioning and developing world. This is the irony of our observation: the professional skill base tends to lead in a direction that many countries are incapable, at present, of following.

Some will argue, quite correctly, that these are countries with limited resources for environmental protection and difficult choices to make. Comparative risk analysis would seem to be an ideal tool under these circumstances. In our view, there is no question of the strength of risk assessment to help policymakers sort through where investments will make the most difference. We are not arguing that some rough forms of analysis are unnecessary. But problems in developing countries tend to be more like the conditions that led to cleanup of the Potomac River sewage problem than they are like the conditions that led to concerns over small residues of chloroform in drinking water. In most cases, a “back of the envelope” analysis, whether of specific or relative risk, will be adequate. That is, instead of a formal risk assessment, it often will be quite adequate simply to identify the most significant sources of environmental pollution and then analyze institutionally and economically viable ways to reduce releases.

Most importantly, the information derived from risk assessment can be illusory without explicit incorporation of the institutional realities within which the decisionmakers work. For example, in the early 1990s, the Peruvian government received warnings about the possible cancer hazard posed by chlorinating drinking water. The government stopped chlorinating the water and a cholera epidemic ensued. Given the existing infrastructure in that country, it was a huge mistake to have tinkered with what was clearly working.

Our concern about the mismatch between intellectual and institutional capabilities in the developing countries leads us to suggest that perhaps “worst things first” is not a sensible doctrine. Principally, these are places where tackling the most difficult problems requires a level of regulatory experience, infrastructure, and government support that simply does not exist. We suggest that in these settings, the pragmatic approach would be to focus on solving lesser problems that will generate regulatory experience and “lessons learned,” to be applied over time to more complex problems. We recognize that this may be less orderly than many would like, but it may well be more effective in the long run.

We also suggest that promoting the use of technical tools such as risk assessment may unknowingly reinforce the tendency in many countries to treat environmental protection as a purely technical exercise engaged in only by experts. A common weakness in the environmental protection regimes of former Soviet bloc countries is a tendency to make decisions about environmental policy based entirely on presumed “science,” with minimal, if any, public outreach. Consequently, little effort is spent determining whether adequate support exists in society for reaching the standards that experts recommend.

Experienced observers of the Russian system of environmental protection, such as Laurence Mee, former coordinator of the Global Environment Facility’s Black Sea Environmental Project, have asked whether Russian priorities stem from a way of thinking that has been in vogue in Russia and the Soviet Union for the last fifty years or so, namely a focus on seeking the technological “quick fix.” Mee contends that Western aid tends to reinforce the idea that everything can be fixed with technology, thus undermining efforts to change public attitudes. From what we have seen of risk analysis done in these contexts, we have to agree.

Policymaking in the countries in transition and the developing world should not occur in a vacuum. In our view, to be effective, risk assessment must be tailored to the circumstances at hand. Experts need to go the extra mile, to contribute more than a narrow technical expertise to the consideration of these critical problems so central to the health and welfare of real people. Without this recognition, risk assessment will be divorced from the genuine needs of the very societies that it seeks to influence.