Benefit Transfer: The Quick, the Dirty, and the Ugly?

"But if you miss, you had better miss very well."

Tuco: "The Good, The Bad, and the Ugly" (1966)

However, it is simply not practical to conduct an original stated or revealed preference research study for each environmental good in each location every time a new policy analysis is conducted. Nor is it really necessary. If someone has already conducted a study that valued a day spent fishing on a small lake in Tennessee, the Army Corps may be able to use that number to value a day spent fishing on a small lake in Kentucky. If someone has already estimated the value of a reduction in cancer risk from decreased exposure to dioxin, the EPA may be able to use that number to value a reduction in cancer risk from decreased exposure to benzene. These are examples of benefit transfer.

The Quick, the Dirty

Benefit transfer is widely used by government agencies because it is quicker and cheaper than conducting original studies every time a benefit estimate is needed. In some cases, benefit transfer is relatively straightforward and even familiar. When the Federal Highway Administration evaluates the time-savings benefits from new highway construction, it does not conduct original research on the value of travel time for every new highway. Rather, it uses per-hour values based on previous studies. Guidelines issued in 1997 suggest a value of $11.90 per hour for intercity personal travel, for example. When the EPA values a decrease in mortality risk from an improvement in air or water quality, it does not conduct original research for each mortality risk. Rather, it uses estimates of the value of a statistical life (VSL) based on previous studies. In the nonroad diesel rule analysis, a VSL of $5.5 million was used.

These are both examples of a benefit transfer technique called unit value transfer. This approach is best suited for situations where the projected impacts of a policy can be measured in fairly homogeneous, divisible units (hours of travel time saved, premature fatalities avoided). A constant benefit value is used per unit of the unpriced good, based on one or more previous valuation studies for that good. Ideally, a unit value is used that was estimated at a study site similar to the policy site. An estimate from a study site located close to the policy site is also generally preferred, to minimize differences between the population of users at the study site and at the policy site. For example, the US Forest Service uses region-specific unit day recreation values as part of their mandated periodic Renewable Resource Planning Act (RPA) Assessment.

The use of unit values may be justified for valuing health impacts, transportation improvements, and some types of outdoor recreation. These are goods that we tend to think of as being more or less homogeneous across users and across policy contexts. Where benefit transfer becomes more difficult is where the context of the good at the policy site differs from that at the study site, either with regard to the attributes of the good being valued or the population enjoying the benefits. An acre of wildlife habitat in Utah is very different from an acre of wildlife habitat in Pennsylvania, and the values generated will likely differ as well.

Value function transfer has the potential to improve the performance of benefit transfer in situations where the good or the user population differs between the study site and the policy site in measurable ways. In this approach, a value function is first estimated at a study site or group of sites. A value function predicts the value of a good as a function of its measurable characteristics (quantity and quality), those of its users (income, etc.), and the context within which the good will be provided (availability of substitutes, etc.). In principle, the value of the good at any policy site can be determined by plugging in the relevant measures for that site. Some have argued that the form of the value function should be motivated by economic theory (Smith, Van Houtven, and Pattanayak, 2002), but more typically it is chosen in an ad hoc manner in an attempt to maximize goodness of fit.

One example of a simple value function is the Army Corps of Engineers' point system for determining user day values for recreation at Corps facilities. Points on a scale from 0 to 100 are awarded for the quality of the site, the number and types of activities enjoyed at the site, and the availability of substitutes for the site. In 2002, user day values ranged from $2.90 for general recreation with low point values to $34.41 for specialized recreation with high point values. Value functions will more often include objective measures of the quality (e.g., catch rate, reservoir size) and measures describing the population of users (e.g., income, travel distance to the site).

Value function transfer will work well only if a) there is sufficient variation at the study site in the attributes of the good, b) there is sufficient variation at the study site in the attributes of the user population, c) the attributes of the good and the population at the policy site fall within the range of the original data at the study site, and d) preferences for the good are similar at the study site and the policy site. One challenge to conducting value function transfer is that original valuation studies are often conducted in a limited geographic area, and important attributes of the good or the population may not vary within an individual study. However, by combining results from several original valuation studies, a value function can be estimated based on a richer set of goods and user populations. In a meta-analysis, value estimates are combined from several different studies. A value function is estimated with these value estimates as the dependent variable and with characteristics of the good, the population of users, and study methodology as the independent variables. For goods where a large number of source studies are available, meta-analysis has the potential to provide value functions that can be applied in a wider range of situations.

While benefit transfer typically tries to tailor value estimates to the policy site's good and population, in some situations it may not be desirable to adjust values to individual contexts. Even though there is some limited empirical evidence that willingness to pay to reduce mortality risk decreases somewhat with age, the EPA chooses to apply a constant VSL regardless of the population at risk. There are important ethical considerations when values are adjusted for age, income, or ethnicity, particularly if those values are used to set policy or to direct resources.

Benefit transfer is clearly feasible only if a study already exists that valued a good similar to the good in question. The analyst must assess the quality of the existing study or studies, and decide whether the good valued at the study site(s) is similar enough to the good at the policy site. The Office of Management and Budget, in its guidance to executive agencies on conducting regulatory analyses (OMB, 2003) provides a common-sense set of criteria that must be satisfied when choosing a source study. The task of finding a suitable source study has been simplified for analysts with the recent creation of databases of previous valuation studies. One of the most comprehensive is the Environmental Valuation Reference Inventory (www.evri.ca) maintained by Environment Canada with support from the United States, Great Britain, and France. Even with such databases, a common problem is lack of documentation in the source studies. Often, this is due to the difference between the information that journal reviewers are looking for and the information that policy analysts need to conduct benefit transfer. For example, a researcher publishing a hedonic pricing analysis will always present the estimated house price function, but might not always report the average house price in the dataset. That kind of information is critical, however, when using that study in a meta-analysis or a benefit transfer.

...and the Ugly?

The first two validity tests are often rejected. That is, it is common to find statistically significant differences between the unit values estimated at two sites and between value functions. While negative results for these tests are informative, they are not necessarily fatal to benefit transfer. With enough data, statistically significant differences can be found even where the values themselves are quite similar. From a policy perspective, the size of the potential transfer error is much more important than statistical convergence.

Regarding the size of the potential transfer error, these studies often find average transfer errors of 40 or 50%, but with a wide range that can span from zero percent to several hundred percent for individual transfer exercises. While generalizations are difficult, there is some evidence that transfer errors tend to be smaller when the two goods are located in the same geographic region (Rosenberger and Phipps, 2001). This may be because the goods themselves are more similar, or it may be because the user populations are more similar. Interestingly, the evidence that value function transfer outperforms unit value transfer is mixed at best. Some studies find an improvement in performance, others do not.

It is important to realize that transfer errors calculated in these validity studies are artificially inflated because the criterion (the value at the policy site) is not perfectly known. Calculated transfer error is the sum of actual transfer error and error in the criterion. Suppose a good has the same value, $100, at two different sites, and that each of the values are estimated at with a standard deviation due to sampling error of $20. A validity test of benefit transfer between these two sites will show an average transfer error of 24%. Compared to this "best case" expected transfer error, an observed transfer error of 40% is not that bad.

...Compared to What?

The answer to the first question depends both on the reason for doing the policy analysis and on the degree to which the value estimate is decisive. Some valuation situations require high precision and reliability. A good example is resource damage assessment, where a responsible party has to write a check based on the value estimate. In contrast, a higher level of uncertainty in the value estimates is probably acceptable when conducting a regulatory impact analysis for a regulation that is mandated by law. Further, the larger the value, both in absolute terms and as a proportion of the total benefit from the policy, the more important it is to get the right number. Finally, value estimates must be more reliable if their uncertainty could potentially tip the balance in favor of or against a proposed action.

With regards to the second question, the uncertainty introduced by benefit transfer may not be large relative to other sources of uncertainty in the value estimates. Estimated values of the same good measured using stated and revealed preference techniques can vary by an order of magnitude. Even within a given technique, research design decisions such as question format or the functional form used for data analysis can have dramatic impacts on value estimates. Indeed, meta-analyses often show that research design features are more important in explaining variation in values than attributes of the goods or the population of users.

The conventional wisdom that an original study is always preferred to a benefit transfer needs to be reexamined. While the potential exists for very large transfer errors, original studies have their own potential for problems. A thoughtful, carefully executed benefit transfer from a high quality, large-sample study conducted at a similar site, or a set of studies conducted at multiple sites, is probably preferable to a small-sample, rushed original study conducted at the policy site.

Nor should the choice between benefit transfer and conducting an original study be necessarily viewed as an either/or choice. Where information on the value of similar goods is available, but there is concern that the value at the policy site may be unique, a Bayesian perspective can be adopted. Value estimates or functions from existing studies can be used to form a prior distribution on the value of the good at the policy site. Valuation research conducted at the policy site provides new information on the value of the good. An updated distribution of the value of the good at the policy site contains information from both previous studies conducted at other sites and from the new research conducted at the policy site. A Bayesian perspective also suggests that the decision whether to conduct original research at the policy site, and if so how much, should be made based on the expected value of the information to be gained and the cost of conducting new research.

Does benefit transfer work? Should we be doing it? The answers to these questions are similar to the answers for nonmarket valuation more generally. Benefit transfer, if done carefully using appropriate sources for the transferred values, can work quite well. However, it can perform very poorly. The same can be said for nonmarket valuation in general. A more constructive discussion is over how to improve benefit transfer protocols and minimize the potential for large transfer error.

And the decision whether we should be doing it has already been made. Federal agencies routinely use benefit transfer to conduct regulatory and program analyses because they have to. If the values of most environmental goods and services are going to be included in these analyses, then they will have to come from benefit transfer. There is not enough time or resources to conduct original studies for each policy analysis. The choice is not between benefit transfer and conducting original studies. The choice, in many cases, is between conducting a benefit transfer and not including any estimate of the benefits from environmental goods and services.

How can benefit transfer be improved? First, the single most important action to improve benefit transfer is to increase the stock of high-quality original valuation studies. With the exception of some types of outdoor recreation and some types of health impacts, the set of available studies for most environmental goods is thin. Second, these studies have to be made available to analysts. Databases like EVRI can serve an important role. Third, the authors of new original studies need to report more details about the methods, data and the good valued. Academic journals tend to discourage publication of study details that are not central to the methodological or theoretical contribution of the research. There have been calls for a new publication outlet, perhaps an online journal, to serve as a repository for this kind of detail. Fourth, the analyst conducting benefit transfer has an obligation to document and justify the assumptions and protocols used. Just as original nonmarket valuation studies must be accompanied by enough documentation to allow judgment of their validity, so too must benefit transfer exercises be transparent and fully documented.

For More Information