Resource Evaluation at the Crossroads

Resources for the Future was founded in 1952. The decade of its beginning was also the decade during which benefit-cost analysis began to receive professional acceptance as a legitimate enterprise for economists seeking to inform public decision making. (That development stemmed from the early work of a small group of economists that demonstrated the linkage between the principles of welfare economics and a practical set of procedures for computing the net benefits arising from public investment projects—primarily water resource projects at that time.) During the three intervening decades, RFF has been a key player not only in the initiation but in the development of benefit-cost analysis as an instrument now used increasingly in the evaluation of many types of public policy issues.

Originally, benefit-cost analysis was applied to traditional public investment projects. Today its applications extend to a diverse array of problems—regulatory decisions, analysis of resource management policies, and natural resource damage assessment being a few examples—the scope of which could not have been envisioned thirty years ago.

This expansion of applications has far-reaching implications for the techniques used and for the treatment of measures of the benefits and costs. Consequently, it has led me to argue for the use of a broader term, resource evaluation, to describe more adequately the amendments and expansions to benefit-cost methods in evaluating today's environmental and natural resource issues. Recognition of the implications of this expansion is especially important today, because the practice of resource evaluation has reached a critical juncture for reasons arising from both political and methodological sources.

Benefit-cost analysis in the evaluation of most national regulatory policies was mandated in 1981 by President Reagan's Executive Order 12291. This mandate also created strong incentives to use benefit-cost analysis in other areas of federal policymaking and in decisions at the state level. And recent environmental legislation has increasingly recognized a need to balance the benefits of new initiatives with their costs. Yet, despite the growing interest in benefit-cost methods for policy analysis, it is reasonable to expect that a new administration will choose to evaluate the achievements of benefit-cost methods and to consider new directions for analyzing regulatory policies.

On the methodological side, much has been learned about the procedures used for measuring benefits and costs of policy actions in carrying out evaluations. Nonetheless, the established practices have been slow to be changed, and they continue to be used despite the fact that many current environmental problems are fundamentally different from the problems that gave rise to existing methods of evaluating public actions. A new codification of theory—and especially of practice—is needed if resource evaluation techniques are to inform decisions that must be made in the future.

Then and now

To provide some perspective on the research issues that face resource economists today, consider the typical public investment decision of the 1950s. It concerned a water resource project. The bulk of the project's outputs were reasonably tangible—water for municipal, industrial, or irrigation purposes; hydroelectric power; improved navigation; or flood control—usually with available market prices or close proxies as valuation measures.

Consequently, project benefits could be monetized using market prices or alternative costs. Project outputs that could not be evaluated in market terms—the so-called intangible items such as the recreation opportunities created by the lakes formed by dams constructed for flood control—were assumed to be unimportant factors in any project. The actions under review, then, were uniformly regarded as improvements; by definition, the development of water resources was in the national interest. The only issue was one of establishing priorities among projects and recognizing the trade-offs between local and national interests in selecting the mix of public works.

In addition, the effects of any one of these projects were principally local. The scale might be large for any specific region but was small in comparison with the national economy. And the time frame was usually limited to one generation of consumers.

Today, the subject of a "typical" resource evaluation is not a project. It is a policy, often a regulation imposing constraints on how private economic activities can be undertaken. For example, we must evaluate the potential levels for national standards for inorganic arsenic emissions or judge the merits of on-board controls for automobiles versus a two-hose system for gasoline refueling to control volatile organic compounds. Of course, water resource projects remain subjects of resource evaluation, but even here the focus of the analysis has changed.

Because today's issues are so diverse, it is difficult to identify a set of attributes common to all of them. Nonetheless, in general, they tend to include several of the following features.

First, public or quasi-public goods, the intangibles of the fifties, now dominate the outputs to be valued. Second, the scale of a policy is acknowledged to be large in relation to the whole economy. Third, the time frame for the effects of a decision can extend over many generations—perhaps even, as in the case of the nuclear waste problem, longer than our records of civilized activities.

A fourth feature of today's issues is that they often involve new dimensions such as technologically sophisticated services (e.g., allocating the use of space for communication satellites), reductions in risk, or complex matters such as the disposal of hazardous materials. Because of the complexities of these issues, the direct experience of households responding to policy analysts' surveys and these respondents' ability to understand and organize the information relevant to a choice may be subject to question.

A fifth characteristic is that, in measuring consumers' values with regard to a particular policy, analysts must provide some means of connecting the expected physical changes in the affected environmental resource to the quality features of the resource that consumers would recognize and value. For example, a reduction in the concentration of polychlorinated biphenyls (PCBs) in the sediment at the bottom of a river, lake, or estuary reduces the prospects for contamination in aquatic life and the potential for bio-concentration, but its direct observability by individuals may be impossible.

Finally, the distribution of effects related to current environmental issues involves not only monetary gains and losses but also physical hazards, with the result that decisions raise important conceptual and ethical issues.

A set of practical guidelines akin to those available for evaluating the water projects of the fifties is clearly needed. While it is recognized that many existing methods for organizing the information inherent in a resource evaluation are inappropriate for dealing with current issues, credible alternatives to "muddling thru" have been difficult to come by. Four major changes seem to offer a start for adapting the methods now in use to fit today's issues.

Assets

Conventional economic practice has dichotomized the analysis of natural and environment resources. The former, whether renewable (forests, water, and so on) or nonrenewable (e.g., minerals), have been treated as natural assets, thereby recognizing the implications of allocation decisions for future resource availability.

By contract, environmental resources (e.g., clean air, clean water, scenic values) have generally not been considered assets. In these cases, attention has been diverted from the resources themselves to the activities involved in using them and to the ways in which the actions of some users affect the well-being of others—for instance, how emissions by manufacturers impose externalities (costs generated but not borne by the producer) in the form of pollution on households. Consequently, the analysis has focused on the harm and associated costs experienced by the households involved. (In principle, these households would be willing to pay at least the equivalent of these costs, and perhaps more, to avoid the damages if a mechanism could be developed for making the payments.)

A needed adaptation of existing methods of resource evaluation, then, is to model both natural and environmental resources as assets, with appropriate recognition of the short- and long-term impacts of allocation decisions. This is important not simply as a reflection of the capacity of the environment to absorb various types of pollutants, but also in attempts to understand and to measure the values that people may place on resources which they themselves do not use.

At a conceptual level, the strategy of treating environmental resources as assets implies that the evaluation of proposed allocation decisions (e.g., public investments) or past allocation decisions (e.g., natural resource damage assessments) involving an environmental resource should be based on how the decision will (or did) affect the value of that resource as an asset and on how much the changes required by the allocation choice will (or did) cost.

Evaluation from this perspective would incorporate a number of considerations: the uncertainty that influences individuals' behavior and their valuing of the resource; use and nonuse motivations that affect individuals' values of environmental resources; and more intangible characteristics of the resource that might be associated with its quality. Such analysis must also consider the time patterns of effects (or outputs) that would follow from decisions so that these effects can be used in gauging h the value of each asset changes with each decision.

Uncertainty

Uncertainty is an integral part of the decisions of economic agents, policymakers, and economic analysts. While there are different types of uncertainty facing each, in sum they have a cascading set of effects on the form and the content of resource evaluations. The constituent sources of these uncertainties have been recognized in past work; nonetheless, there is still no consistent treatment of all of them. Moreover, except in rather special cases, there appear to be no practical guidelines for policymakers on how to effectively incorporate the implications of these uncertainties into policy decisions.

With regard to economic agents: since uncertainty exists in all decisions made by firms and households, from the perspective of the analyst describing the behavior of these agents, what is relevant is whether the uncertainty is "important enough" to affect what they do. There has been long0standing recognition of this point in economics, but slow replacement of the conventional practices with methods that adequately incorporate the implications of uncertainty for economic behavior and the corresponding measures of the resulting benefits and costs.

Uncertainty in policymaking takes several forms. Policies are fashioned in an uncertain environment. They may well deliver uncertain results. And, most recently, with an important class of environmental initiatives—the Environmental Protection Agency's risk management policies—they have been focused on actions that are intended to reduce some of the uncertainty that faces individuals.

The theory and most especially the practice underlying resource evaluations have not adjusted to the fact that in some cases economic agents' behavior is in response to the uncertainties they face. Consequently, in measuring the changes in individuals' welfare that would accompany resource allocation decisions, people may not be guaranteed a level of well-being. Instead, policy may simply reduce the chances of undesirable outcomes, not guaranteeing safety but merely improving the odds for a safer environment.

The uncertainties facing the economic analyst are also varied. The majority of economic analysis, as it is currently applied to the testing of hypotheses or estimating models, relies on outcomes after choices have been made. Observed choices are supplemented with assumptions about what was known by the individual (or firm) pr to the observed outcome and with postulated constraints that are treated as relevant for the decision, as well as with a set of assumptions describing how both were used in the choice process.

These models provide the basis for the indirect methods of estimating households' values for environmental purposes. There is, however, no objective indicator for uncertainty about any of the elements facing the household prior to the choice. There are no mechanisms allowing individuals' perceptions to be "signaled" to the analyst.

In response to these limitations, resource economists have begun to use survey techniques to learn about how individuals respond to hypothetical choices. The direct, or contingent valuation, method for estimating the values that individuals place on improvements in specific aspects of environmental resources (or on risk reductions) is one example of these efforts. A full understanding of the strengths and limitations of these methods will require a model of how individuals perceive and verbally respond to the ways in which analysts describe choice situations to them.

Clearly, such an effort extends beyond the confines of economics to psychology and the other social sciences. A new model incorporating a full range of social science insights for decisionmaking that involves new or uncertain choices is warranted here. Economics can provide a basic structure to begin the process, but to complete the framework will require incorporating the insights of psychology and sociology.

Complementary use of direct and indirect valuation methods (surveys and market-related measures, respectively) can reduce the uncertainty the analyst faces in estimating the values for resources. These joint applications could involve collecting information that allows estimate of individuals' values for environment resources as well as their demands for their goods or services supplied by the market. By comparing the demands for marketed goods with results derived from other sources, it is possible to gauge how individuals perceive the constraints described to them in a survey format. This comparison may allow analysts to gauge how perceptions are formed with surveys involving valuation questions for nonmarketed resources.

Scale

In contrast to the project orientation of the 1950s, most resource-related applications of benefit-cost analysis today involve situations in which decisions are likely to affect the national economy. The wide-ranging effects of a policy decision in such cases can be purely economic, working through markets, or they can involve a large component of the ecosystem and hence the environment of several nations simultaneously.

The extent of change in the scope of resource issues over the years—and the corresponding expansion of the implications of policy decisions—are easily illustrated. Executive Order 12291 requires that benefit-cost analyses be performed for regulations that are expected to have major effects on the economy (i.e., annual impacts of at least $100 million). Thus, the very screening criteria require benefit-cost analyses for cases where the assumptions of small scale are likely to be violated. Equally important, several of today's resource problems resulting from human activities—acid deposition, climate change, sea-level rise, and marine pollution—are affecting the environment on a global scale.

As the scope of the problems and of related policy decisions has broadened, the range of issues to be considered in evaluating any individual policy has also expanded. Scale now matters. Policies no longer exist in isolation. Their mutual interaction is important (although this is not universally appreciated in the current body of literature on resource evaluations).

A recognition that scale changes the types of assumptions that can reasonably be made in developing resource evaluations raises a number of important issues. For instance, it is more difficult to estimate individual values for resource changes when these changes affect other commodities that people value.

Before practical methods can be developed for taking account of scale, several questions must be answered. Even when resource evaluations can use indirect methods for valuing commodities (i.e., market prices as yardsticks for assigning values), under what conditions can the indirect effects of large-scale price of regulatory changes be expected to affect the approaches used to measure the values of both marketed and nonmarketed commodities? Clearly, the answer reveals something that has sensitive the indirect methods for valuing non-marketed commodities would be the effects of scale.

Nor does he use of direct methods—i.e., surveys—for valuing commodities automatically provide solutions. A major question here is whether it is possible to determine the conditions that survey respondents assume will affect them in their other decisions. Their reported valuations for environmental resources will be affected by their perceptions of the constraints facing them. Learning what respondents assume in giving their answers seems to offer a more promising strategy than framing survey questions so they include all possible side conditions to respondents' decisions.

The long time scale associated with many resource evaluations of today also presents new challenges for resource evaluation. Regulation of chlorofluorocarbon emissions, policies on carbon-based fuels, and selection of a site for the long-term repository of commercial nuclear waste, for example, involve long-term effect crucial to the analysis.

Conventional practice has been to use present value as the basis for summarizing streams of costs and benefits over time. Since the issues under analysis have often been cases where benefits are conveyed to the current generation and costs imposed on the future, the timing of when costs and benefits are realized and the selection of the rate converting future dollars into current "equivalents" have been crucial to the use of conventional methods.

Which research issues might reduce the scope of conflicts involved in making decisions with effects over such long time spans? At least two deserve further evaluation. The first involved a recognition that the conflicts are greatest where there is uncertainty over and some degree of irreversibility in the outcomes of decisions.

The second research issue involves a different type of irreversibility—the inability to change the sequence of indirect effects that accompany large-scale decisions. For instance, suppose that a public investment decision which increases access to coastal areas for recreationists must be evaluated; it is known that these areas will be inundated as a result of progressive sea-level rise over the next fifty to seventy-five years. The enhanced access resulting from the investment will promote the development of second homes, private recreational facilities, and complementary public infrastructure—all of which ultimately increases the cost of a sea-level rise in the future.

In many respects, the problem is akin to development in a flood plain after public investments are made to reduce the likelihood of flooding—the costs of subsequent floods increase dramatically. In this case, however, there is one important difference. As knowledge of the causes of temperature change increases, the timing of change in sea level becomes more predictable. Thus, climate change is different from the case of floods or coastal hurricanes, here, after public action to reduce the prospects for flooding there is often sufficient diversity of opinion to permit the existence of insurance markets.

This difference implies that private indirect effects must be treated differently. To the extent that there are limits on the ex post losses that society will allow private citizens to experience, it is especially important to recognize how the secondary effects of current public investments could serve to increase future costs.

Evaluation

Finally, policy research itself should not be free from evaluation. As experience is accumulated in valuing nonmarketed resources, analysts must learn how to learn from that research and integrate the findings into improved use of what is on the proverbial "research shelf."

The mandating of benefit-cost analysis has created a "procedural" demand for literature reviews. Moreover, much of the legislation leading to this demand has at the same time precluded new estimates, advocating instead the use of the "best available" findings. At present, learning from the record has become a procedural step in the development of a regulatory impact analysis, in much the same way that the environmental impact statement of the National Environmental Policy Act has typically been a procedural document without insight into the potential environmental problems involved in any specific decision.

These criticisms are especially relevant to reviews of studies of the valuation of nonmarket resources. Although there is a growing body of findings from such studies—models used to estimate the marginal values of air pollutants, travel-cost/recreation-demand models for gauging the value of a variety of recreational resources, wage models for the valuation of risk changes, and direct surveys for the values of a range of environmental resources—analysts do not know how to systematically learn from them. Such learning would result in a better match between off-the-shelf estimates when they are applied to new valuation problems as well as a better understanding of the research needed to make what is on the shelf more useful.

Defective policymaking?

Resource economists, especially those associated with the use of applied welfare methods to address policy issues, have increasingly been questioned on their "naive view" of policymaking. Critics argue that the emergence of applied welfare economics is actually a story of manipulation, that those interested in water projects used the analysis to meet their predefined objectives. In the water resources area, they say, there was a social consensus favoring public works in water development. Benefit-cost analyses were simply gauges of the feasibility of projects and of the implications of small design changes. At most they served to set priorities among a slate of projects to be undertaken, never to decide whether such projects were desirable.

These are not new arguments. Benefit-cost analysis was never intended to be the exclusive basis for decisionmaking. However, the more analysts can narrow the sources of error in the use of methods over which, admittedly, reasonable people can disagree, the greater the precision of our measuring rod.

Opportunities for manipulation are reduced by improvements in methods and consensus over best practice. This allows the terms of the debate to be more clearly defined. Decisions will always imply values for the outputs and interests served by them. What is at issue is defining a standard that allows these values to be compared.

The decade of the seventies was one of stating broad environmental and resource goals—clean air and clean water, preserving our national heritage with wilderness and wildlife preserves, and reducing energy dependencies. At the outset of the decade, it was assumed that most of these goals could be achieved with reasonable costs and thus, if conflicts arose, they would not be large.

In the eighties, we have learned that this assumption was wrong. As policies required to meet some of the goals began to be implemented, there was a greater appreciation of the large costs and growing conflicts over the use of natural and environmental resources. Not everyone can win; some people will have the nuclear waste repository in their backyard. Air and water quality are unlikely to be uniformly clean nationwide. Risks will be unequally distributed. Arbitrating the conflicts that naturally precede the decisions on each of these issues requires a standard with which to compare those whose interests are served with those who experience a disproportionate share of the costs.

In the nineties, we will be forced to make increasingly difficult decisions over resource use. The quality of those decisions, as well as the ability to make them in a reasonably democratic society, requires the ability to compare consequences. Resource evaluation methods can provide this standard—but the methods and their application must meed the demands of the problem at hand.