Designing the Electricity Markets of the Future, with Chiara Lo Prete

Notable Quotes

• Regulations distinguish electricity markets from other markets: “Airlines can set very low prices to fill flights when demand is low and extremely high prices when demand is high. This doesn’t happen in electricity markets, because electricity markets have offer caps that limit a supplier’s offer price into the market, and in many cases, they have price caps that limit the level of the market-clearing prices. Why were these caps introduced? Primarily to limit the ability of the suppliers to exercise unilateral market power in the short-term energy market.” (12:39)
• Renewable energy may not be able to provide power in a pinch: “The key challenge [with the transition toward carbon-free electricity generation] is that the amount of energy that is supplied by renewable resources during high-demand periods can be unexpectedly low. This implies that potential challenges to grid reliability extend beyond just peak hours, as it used to be in the past.” (16:56)
• Predicting electricity demand is becoming more difficult: “Constructing load forecasts is going to become more and more challenging, not just for the United States, but for other regions, as well. The United States and other regions anticipate historically high rates of electricity-demand growth due to a combination of factors like growing demand from data centers and electrification of buildings and transport.” (30:12)

Top of the Stack

• “Time for a Market Upgrade? A Review of Wholesale Electricity Market Designs for the Future” by Chiara Lo Prete, Karen Palmer, and Molly Robertson
• Superpower: One Man’s Quest to Transform American Energy by Russell Gold
• 4 3 2 1 by Paul Auster

The Full Transcript

Kristin Hayes: Hello, and welcome to Resources Radio, a weekly podcast from Resources for the Future. I'm your host, Kristin Hayes. My guest today is Dr. Chiara Lo Prete, an associate professor of energy economics in the John and Willie Leone Family Department of Energy and Mineral Engineering at Penn State University; that is among her many appointments. Her research centers on the economics of energy markets, and she's particularly interested in the competition and design of electricity markets, which is going to be the topic of our conversation today.

Another one of her very intriguing research interests is around the weaponization of electricity trade. We're definitely going to have Chiara back on the show in the future to talk about that, but for now, Chiara and I are going to discuss some work that she's recently completed with two of my Resources for the Future (RFF) colleagues, Karen Palmer and Molly Robertson, looking at wholesale electricity market designs and resource adequacy under a decarbonizing grid. I promise we will explain all of that. So stay with us.

Hi Chiara, welcome to Resources Radio. It's great to talk with you today.

Chiara Lo Prete: Thanks, Kristen. It's great to be here.

Kristin Hayes: Great. I skimmed over some of your many diverse interests and how they’ve evolved in your career, so let me turn it back over to you to kick us off. I'd love to hear a little bit more about your journey to working on electricity and electricity markets. How did that interest start for you, and how has it evolved over the course of your career?

Chiara Lo Prete: I'd say that my journey has been the result of a fortuitous combination of circumstances; that's what I would call it. I graduated from college with a degree in economics in Italy and then started applying for jobs, like many recent college graduates who still don't quite know what they want to do with their life. I wasn't really considering grad school at the time, and I remember submitting many applications to a number of companies, including consulting firms, accounting firms, and investment banks. Again—nothing fundamentally different from what happens to many college graduates.

Then, one day my dad came to me and showed me a fellowship ad in a newspaper. The fellowship said that Eni, which is a big, integrated energy company, was offering about 20 or 25 fellowships to attend a one-year master's degree focusing on energy and the environment at a very prestigious school in Milan called Scuola Mattei.

This school has been training energy-sector professionals from both Italy and overseas since the 1960s. It's been going on for a long time. It sounded like a great opportunity to learn about an exciting field, so I applied. I didn't have any knowledge about energy at the time, and the number of fellowships was admittedly low, so I didn't think I had a good chance. But I got one of the fellowships, and I moved to Milan. I would say that that year for me was truly transformative, because it really got me hooked on energy economics.

At the end of the program I was hired as an energy analyst in Eni's office of energy research. I worked there for a couple of years, and I would say that's what set me on the path to energy modeling. When I moved to the United States for my PhD, I had the great fortune of working with some really great mentors in highly interdisciplinary departments in schools where I specialized in power sector modeling. I started working on issues related to electricity markets, and that's where I've been doing most of my work since.

Kristin Hayes: Excellent. Sorry to be a little cheesy, but I love the joy that's in your voice when you recreate that fortuitous turn of events. It sounds like it's worked out very well. I'm very glad it's brought you here today.

Let's talk about wholesale market design. We're going to cover a range of somewhat weedy, but very important issues today. This new research focuses on new market designs to ensure resource adequacy for the grid of the future. Let me start with a natural first question: What exactly do you mean when you say resource adequacy?

Chiara Lo Prete: That's a great question. At a very high level, resource adequacy is a measure of whether there are sufficient resources such as generation assets that are available to meet electricity demand while accounting for uncertainty in generation; for example, outages and demand uncertainty—I'm referring to the uncertainty in the forecasts. Now, historically, resource adequacy focused on capacity over peak time periods. Specifically, resource adequacy was assessed based on capacity-reserve levels compared to peak demand, because resources were generally dispatchable and available when needed except for again, unit outages. What was happening was that reserve margins were planned, so the deficiency in capacity to meet daily peak demand occurred no more than one day in 10 years. This is the standard that we hear about very often.

I want to underline here that a key assumption in all of this has always been that fuel is available when capacity is required to provide the needed energy. This assumption was because fuel was available for a long time either through long-term fuel contracts or on-site storage.

Kristin Hayes: That's a great lead into my next question, which is about what the status quo has looked like for ensuring resource adequacy today. Forget the grid of the future for just a second, but in the grid of the present, what exactly are either system operators or generators doing to ensure resource adequacy in the current moment? Reflect just a bit on whether that strategy has been successful. We all have our personal recollections of when that strategy has not been successful, but I'd love your take on a broader and more holistic level on whether things are working now.

Chiara Lo Prete: I'll start by saying that things are very different around the world, and we focused on the United States, so I'll just try to focus on the United States in my remarks today. I'll say that in the United States, resource adequacy has been tackled in, I would say, four ways. The first one is that, as many of our listeners know, some states deregulated electricity generation over the past 25 years. Despite that, electric utility companies in other states today remain regulated monopolies. Major parts of the United States such as the West (essentially excluding California and the Southeast), operate under this traditionally regulated monopoly model. Vertically integrated utilities that operate in these markets procure resources centrally. They have what is called a centralized procurement framework.

Moving on to the second type of way, I'm going to focus on California. California is one of the regions that has a decentralized procurement framework. That means if some shortfalls are predicted, then the state's resource-adequacy rules and guidance would prompt the load-serving entities to go through a procurement process to essentially purchase more generation capacity to close that gap.

The third method is something that is closer to where I live in Pennsylvania, which is part of PJM Interconnection. PJM is the entity that coordinates the movement of electricity through all, or parts of, 13 states and the District of Columbia. PJM, like other regions in the United States, has a capacity market that operates before the energy market. How does that work?

Again, at a very high level, PJM requires the load-serving entities to purchase quantities of capacity that the operator has determined will be sufficient to meet peak demand plus a certain reserve margin. That is typically around 15 percent. In this market, generators bid a specific quantity of capacity at a price, and bidding into the capacity market means obligating your resource to deliver electricity or otherwise risk being subject to some penalties if you fail to deliver when you're called on.

The last approach, number four, to try to tackle resource adequacy relies not on a different market, but rather on improving price signals in the energy market. This is the so-called scarcity-pricing approach that was adopted by the Electricity Reliability Council of Texas (ERCOT). ERCOT reformed its energy market a few years ago so that peak energy prices would reflect scarcity of capacity in a better way. One thing to note is that over the past couple of years, there's been a host of very controversial energy policy bills that made their way through the Texas legislature; one of these very controversial bills was an $18 billion out-of-market directive to build up to, I think it was, 10 gigawatts of new natural gas–fired plants that would just be sitting in reserve. This is kind of going in a slightly different direction than the scarcity-pricing mechanism that I described. But again, ERCOT is the only one that doesn't have a formal long-term resource-adequacy framework like the others.

You could ask—which I think is a really interesting thing that many folks don't think about—what is different about electricity markets that requires the need for a long-term resource-adequacy mechanism? Consumers obviously want to be able to draw electricity from the grid when they need it, but that's true for other goods and services, right? Think about air travel, for example. We want air travel, but consumers do not pay for airplanes. There is no regulatory intervention that ensures there is sufficient production capacity to meet demand for air travel. So, why is there a difference? I would say that the difference lies in how short-term markets for air travel (and other products, of course), operate relative to the wholesale electricity market. In the markets for air travel, there's really no explicit prohibition on the short-term price rising to a very high level to clear the market, right?

Airlines can set very low prices to fill flights when demand is low and extremely high prices when demand is high. This doesn't happen in electricity markets, because electricity markets have offer caps that limit a supplier's offer price into the market, and in many cases, they have price caps that limit the level of the market-clearing prices. Why were these caps introduced? Primarily to limit the ability of the suppliers to exercise unilateral market power in the short-term energy market. That was the intent, but, of course, introducing the caps really had two effects.

The first effect was that this reduced the revenues that suppliers receive during scarcity conditions, something that is referred to in the literature as the “missing money problem.” The second issue is that the caps create big incentives for the electricity retailers and the consumers to underprocure their expected energy needs in the forward market and delay their purchases of electricity until real-time operations. This can result in energy shortfalls during high demand conditions and expose customers to extremely high prices for sustained periods of time. All of what I just described has been termed a “reliability externality,” and this is the reason why we need a regulatory intervention to internalize.

Kristin Hayes: This is fascinating. I'm learning a ton, and I think particularly your explanation of the heterogeneity of types of markets that exist in the United States already lays a really good foundation for our conversation about the work that you've been doing on new market designs. They're being layered on top of a very heterogeneous system, a system that has tried different approaches already.

Let's pivot and look ahead to the work and your consideration of new market designs, specifically in the context of decarbonizing the grid. You mentioned at the beginning how some of the current systems have really relied on very consistent and reliable access to fuel sources, and that's something that may change a little bit in a cleaner grid. I'd like to invite you to say a little bit about what it is about this transition away from fossil fuel generation, or other types of very predictable generation, that is prompting some of the thinking about these new market designs.

Chiara Lo Prete: That's a great question. I would start by saying that I see two overlapping energy transitions. One is more in the short term, the other one is a little bit longer term. The short-term energy transition is the one from, say, coal to gas, which obviously leaves the US power grids increasingly reliant on natural gas generation. A wide array of organizations have produced various scenarios that unanimously foresee a continued role for natural gas–fired generation as a balancing resource going forward. Now, unlike wind and solar, which are intermittent, natural gas is a dispatchable resource but in some cases in the recent past has proven to be a liability, especially under extreme winter conditions. An example is what happened during winter storms in Texas and Elliot in PJM, when natural gas generation saw very high rates of unavailability in ERCOT and PJM respectively.

The second transition is more of a long-term transition, and that's the transition toward carbon-free generation. The key challenge there is that the amount of energy that is supplied by renewable resources during high-demand periods can be unexpectedly low. This implies that potential challenges to grid reliability extend beyond just peak hours, as it used to be in the past. It's really about all hours of the year, because reliability challenges happen now more and more often during hours of peak net demand.

Net demand is the difference between the total electricity demand and the amount of electricity generated by renewable energy. Again, there's not a focus on peak demand anymore, as it used to be, but it's more about peak net demand. That's the reason why there's been more emphasis on the availability of capacity to supply energy to serve net demand—not just capacity in general, but capacity to serve net demand.

Is there adequate energy available to serve realized demand during all hours of the year? Well, grid operators, in recent events that we've witnessed, were often required to shift to curtailing electricity demand to maintain balance during very high-demand periods. Again, an example is what happened during Winter Storm Uri in Texas in February 2021, but Texas is not alone in this. In summer of 2020, California's grid operator, which is the California Independent System Operator, had to initiate rolling blackouts to preserve system stability in the midst of a multiday heat wave. Again in 2022, there was another heat wave in which the California Independent System Operator had to curtail load and narrowly avoided blackouts.

You see here that Texas, like we said at the beginning, doesn't have a formal long-term resource-adequacy mechanism in place, although it did have a very high offer cap on the short-term market in 2021.

California, on the other hand, relies on a capacity-based approach to address the reliability externality. Both of these systems have very significant shares of intermittent renewables. Under the scarcity events that I just described, the fundamental challenge was not having adequate generation capacity, but rather having adequate energy that was available to serve demand when it was needed.

To answer your question on how effective these mechanisms have been, I think there is a sense that traditional methods for ensuring resource adequacy have really not done enough to actually guarantee that supply will be available in times of system needs. This has prompted thinking about new market approaches that do not require a complete overhaul of electricity market design.

Just to give you one example, a number of electricity market operators are considering seasonal forward-capacity markets where resources would bid different quantities and prices based on their projected seasonal availability. In the paper we do discuss some of these new market approaches that grid operators are thinking about to tackle the challenges that we've been talking about.

Kristin Hayes: Again, it's fascinating, and I'm learning so much. It sounds like—correct me if I'm wrong—you also looked at some proposals in addition to the seasonal capacity market that you just mentioned. I feel like I've heard or talked with you separately about how that is an evolution of the current market, but it's not a radically new way of approaching some of these market challenges. It sounds like there are also some proposals that you looked at that are sort of pushing the envelope a little bit more, that are a bit more expansive in how they're thinking about addressing these challenges.

That's the first thing I'd like to clarify with you; what are people working toward now? Are they nibbling around the edges, or are people really starting to branch out of the box in thinking here? In terms of that out-of-the-box thinking, I know you looked at a range of different proposals; I'd love to know a little bit more about the variety of ways that people are thinking about solving these problems. I know there's going to be a lot to cover, but if you could give us an example of just one or two of those sort of market designs that are being considered, that would be great.

Chiara Lo Prete: First, just like you said, the core of the paper with Karen and Molly is a review of 11 proposed electricity market designs that try to tackle various challenges associated with the clean energy transition, including, but definitely not limited to, resource adequacy. These 11 proposals are in varying stages of development, and they have yet to be tested in the real world.

We divide them into five groups. Just to give you a very high-level kind of description of the five groups, we begin our discussion in the paper with a design that proposes the combined procurement of forward capacity and clean energy attribute credits. This is essentially a variation on existing capacity markets and probably the least radical departure from current market designs.

Then, we move to designs that adopt a very different approach to do different things. For example, some proposals feature long-term contract auctions and energy markets. Other proposals look at centralized resource planning that in some cases may be coupled with bidding. Other proposals look at separate short-term markets for different generation technologies, renewables and nonrenewables. The last proposal in our review is a design that thinks about a collection of what are called linked-swing contract markets. These swing contracts are intended to ensure the availability of net load-balancing services for real-time operations.

The swing contract design is definitely the most fundamental shift in electricity market operations. It would require a pretty major change in the way electricity markets operate today.

To your second question, which was how they address resource-adequacy challenges, there's definitely a lot of variety in the designs that we reviewed, and I just want to give you a few examples.

One of the designs requires the electricity demand in the system to be covered by what are called fixed-price forward contracts. Who holds these forward contracts? The electricity retailers. They're required to hold hourly forward contracts for energy for fractions of realized system demand at various horizons for delivery. They are on the demand side, and they enter into the forward contracts with electricity suppliers who have very strong financial incentives to find the least-cost mix of resources to meet their forward-contract obligations.

One of the key points of this design is that because retailers are protected from high short-term prices by the fact that contract holdings are equal to electricity demand, the offer cap on the short-term energy market can be raised to increase the incentive to produce as much energy as possible during stress-time conditions.

This is kind of the reason why in this forward-contract design, suppliers would be incentivized primarily through, again, the high price that is observed in the short-term market. They would be incentivized to be available and to produce in the real-time market. Another approach supports resource adequacy in a completely different way, and it does so by centrally selecting the optimal mix of investments based on very granular regional data of renewable availability across many hours, many years, and under a variety of weather conditions. In this case, there's a big optimization approach that centrally determines the optimal mix of investments.

Another approach, the last one that I'll give you, tries to foster resource adequacy by offering remuneration ex post. Resources are paid for ex post verified performance. If they perform, they receive an ex post payment. Obviously, this provides incentives to be available and follow the dispatch signals of the grid operator. As you can see, I just talked about three proposals that adopt very, very different approaches to tackle that one challenge that we've been focusing on today.

Kristin Hayes: This is such a great illustration of the complexity of what underlies, what for most of us is, a very opaque process of flipping on a light switch. Either the power comes on or it doesn't, but it really is a great illustration of the subtleties, the financial instruments, and the complexity of these markets. I think it's really good for people to hear a little bit more about the depth of thinking that has gone into some of these proposals too.

Chiara, I know we're getting close to our end of time. I'm going to ask you a summary question.

There are obviously a number of creative ways that decisionmakers in this space, whether those are policymakers or folks within the operational infrastructure of the grid, choose ways that they can address the challenges that you've been describing. After you've pulled all this information together, looked at these designs, and gauged them against the criteria that you've identified—and I would encourage our listeners to go take a look at those criteria, because there's some really interesting stuff there, too—what would you call out? What's in your mind after this review that could really help those decisionmakers decide what path to take?

Chiara Lo Prete: I think the different stages of development and also the different points of emphasis of the various designs we reviewed make it kind of difficult to identify a specific recommendation for policy. What we did do in the paper is highlight several open questions and directions for future research.

I just want to mention one here that I think is really interesting. Like I said, several proposals require load-serving entities to procure energy to meet their demand years ahead of electricity delivery; for example, through forward contracts. I would say one approach in the proposals is to define mandatory forward retailer obligations based on load forecasts at different look-ahead horizons. For example, I could ask retailers to hold 85 percent of forecast demand four years in advance of delivery and maybe 95 percent of forecast demand one year before delivery. But the key challenge here is that the determination of these mandatory energy-purchase obligations and also the enforcement of these obligations are really far from obvious and are definitely a point that we thought deserved closer scrutiny.

One of the challenges is that constructing load forecasts is going to become more and more challenging, not just for the United States, but for other regions, as well. The United States and other regions anticipate historically high rates of electricity-demand growth due to a combination of factors like growing demand from data centers and electrification of buildings and transport. The impact of these factors on total demand and load shapes is definitely uncertain and forward auctions that try to procure a substantial fraction of energy years ahead of time may result in energy overprocurement or underprocurement, and therefore, they may require significant adjustments exposed if 100 percent of that electricity demand must be covered by forward contracts.

Kristin Hayes: Right. Well, I'm sorry that we're out of time to talk about this, because I do feel like there's so much in here. Once again, as always, I will point our listeners to the document that underpins this conversation for the rich detail here.

I want to close with our regular feature, Top of the Stack. Chiara, let me turn the last word back to you again to share some recommendations of good content on this topic or on other topics to share with our listening audience. So, what's on the top of your stack?

Chiara Lo Prete: I have a couple of books. One is a nonfiction book that actually came out a few years ago, and that's called Superpower by Russell Gold, who's an energy reporter at the Wall Street Journal. Although the book is not new, I recently learned about it while I was attending the Austin Electricity Conference this year. The conference featured a keynote by Michael Skelly, who is an entrepreneur who began working on wind energy 25 years ago when many considered the industry essentially a joke. What's fascinating about the book is that it describes the evolution of the wind energy industry through the eyes of Michael Skelly, who absolutely wanted to build transmission lines to connect wind resources in the panhandle and a potential customer, which ended up being Tennessee Valley Authority. I think the book does a really good job at describing the electric power industry and also key milestones of the industry like the Public Utility Regulatory Policies Act and other things. It conveys the scientific parts in very accessible terms and simple analogies, so I would definitely recommend it. It also has a number of very interesting characters, just from a personal point of view. Michael Skelly is one of them for sure.

Kristin Hayes: I'm sure.

Chiara Lo Prete: The other book that I would like to recommend—I'm not done, I'm about halfway through—has nothing to do with energy, but it's an epic novel by Paul Auster, who unfortunately passed away just a couple of months ago. In general, I've been a big fan of Paul Auster over the years. I think I've read most of his books. The book that I'm referring to is called 4 3 2 1, and it presents the life of a boy called Archie Ferguson in four versions simultaneously.

Essentially, Auster sets all four of his stories on parallel tracks and tells them simultaneously. There's four versions of chapter one, four versions of chapter two, and so on. That makes it a little bit difficult and maybe complex to follow, but what I think is absolutely fascinating is that what changes with each scenario are the people that Archie comes in contact with and the influence that they exert on his life. This aspect is something that really resonated with me and it kind of gave me an opportunity to think about and reflect on my own path and the people that I met who definitely influenced my own journey, both professional and personal.

Kristin Hayes: Fantastic. I really love that as a conceptual framework for a book. That's a great recommendation. I appreciate it.

Thank you again for talking through these complicated issues with us in a marvelous way. It illustrates the nuances, but really makes it easy to understand too. I really appreciate that and I look forward to staying in touch.

Chiara Lo Prete: Thank you so much, Kristin. Thank you.

Kristin Hayes: You've been listening to Resources Radio, a podcast from Resources for the Future, or RFF. If you have a minute, we'd really appreciate you leaving us a rating or a comment on your podcast platform of choice. Also, feel free to send us your suggestions for future episodes.

This podcast is made possible with the generous financial support of our listeners. You can help us continue producing these kinds of discussions on the topics that you care about by making a donation to Resources for the Future online at rff.org/donate.

RFF is an independent, nonprofit research institution in Washington, DC. Our mission is to improve environmental, energy, and natural resource decisions through impartial economic research and policy engagement. The views expressed on this podcast are solely those of the podcast guests and may differ from those of RFF experts, its officers, or its directors. RFF does not take positions on specific legislative proposals.

Resources Radio is produced by Elizabeth Wason, with music by Daniel Raimi. Join us next week for another episode.