EPA’s Clean Power Plan uses expanded energy efficiency programs as a component of states’ emissions rate targets. States that choose to use energy efficiency for compliance need to develop and provide EPA with a plan for evaluating energy savings that result from the policy. In the technical support document for state plans, EPA describes the state of the art with respect to Evaluation, Measurement & Verification (EM&V) of energy efficiency programs and suggests a number of approaches that states might adopt.
EPA’s discussion of EM&V focuses on traditional engineering-based methods. These calculations are sometimes (but not always) adjusted to reflect the fact that some of the consumers who participate in an efficiency program may have made the investments anyway. When they are made, adjustments are based on surveys that ask customers whether they would have invested without the program, a method of questionable reliability. The engineering approaches also may fail to account for the interactions between efficiency enhancements related to one end use and energy consumption for another end use. (For example, replacing incandescent lights with cooler compact fluorescent lights or LED lamps could increase demand for energy for heating in the winter and reduce demand for energy for cooling in the summer.) They also fail to account for the so-called rebound effect, an increase in usage that may occur when efficiency improves. And the engineering approach is not well suited to policies that work through behavioral “nudges,” information provision, and other non-technology based approaches.
An alternative approach to evaluating energy savings would be to use actual customer-level energy consumption data, comparing energy consumption before and after a policy takes place for those affected and a control group., This approach eliminates the need for a separate net to gross calculation and it automatically accounts for impacts of the efficiency policy across different energy end uses. And the approach can be used for nudges, information provision, and similar policies. This econometrics approach is often used in the scholarly economics literature, but typically has not found its way into mainstream energy efficiency EM&V.
The development of compliance plans for the Clean Power Plan provides an opportunity for states and utilities to experiment with these more robust evaluation methods. Compliance with EPA’s proposal is not required until the 2020-2029 period, so there is time for utilities and states to start experimenting with new evaluation methods. As these experiments unfold we can start to build a base of knowledge that will enable better forecasts of future energy savings and energy efficiency potential, better targeting of energy efficiency resources and ultimately more cost-effective policies for saving energy and reducing carbon emissions.
This type of empirical analysis can be challenging to do because access to customer level energy use data is hard to come by and identifying a relevant control group can be a challenge. However, neither challenge is insurmountable.
One way to solve the first challenge is to randomly assign customers to the energy efficiency intervention. This is the approach that is taken in the Home Energy Report program (operated by OPower). Under this program, electric utilities sign up to have Opower send randomly selected customers reports on their home energy use and how it compares with that of other similar households. When energy consumption before and after receiving a report for households that receive the report is compared to the change in energy consumption for households in the control group, studies find that simply providing the reports produces a roughly 2 percent reduction in energy use.
When random assignment is impossible, another approach would be to provide a randomly selected group of customers extra encouragement to participate in an energy efficiency program and then make similar comparisons in energy use before and after the program takes effect between affected and encouraged customers and others.
A third approach would be to build in eligibility requirements for program participation (limited time offers, limited program budget, size thresholds for eligibility) that provide a quasi-experimental dimension and facilitate the creation of a good control group. With sufficient customer level data, such as that used by Lucas Davis, Alan Fuchs and Paul Gertler in their study of the Mexican Cash for Coolers program, customers participating in an efficiency program can be matched with closely associated control households for clean identification of the policy’s effect.
The second challenge of obtaining access to customer level energy bill data is a long standing one reflecting utility and customer concerns about customer privacy that have increased with the widespread introduction of smart meters that collect a large amount of data on real time energy use. However, this concern could be addressed by regulators requiring utilities to make data available to evaluators and researchers under strict non-disclosure agreements and privacy protections. Economics researchers who are experts in using these methods have experience with such agreements and have procedures for protecting data confidentiality. Adhering to these practices and procedures is in the researchers’ interests because they are always looking to the next research project and getting access to the next data set will require good stewardship of the current one.
The types of evaluations discussed here would provide a cleaner and more robust picture of the net effects of efficiency policies and programs on overall customer energy use in homes and commercial buildings than we currently have. Impacts measured using these approaches provide a better basis for assessing the CO2 emissions reductions that actually result from efficiency interventions and that might be expected in the future. Using state of the art approaches to evaluating efficiency policies is an essential ingredient to finding the most effective and cost effective approaches to reducing energy consumption and CO2 emissions.
