Hot chips, algorithms and electricity markets
It's all connected
In a wholesale electricity market, competition is key to reducing electricity costs. A more competitive market will push down the profits electricity companies can earn. Inversely, an uncompetitive electricity market leads to consumers getting ripped off.
A properly competitive wholesale electricity market can be a hard thing to gauge for an average punter. It is, however, much easier to observe the distaste for the outcomes of uncompetitive markets - one only needs to recall the reactions to Qantas or our supermarket oligopoly.
As energy markets globally shift towards renewable power systems, market forces are fundamental for managing prices through uncertain times. Amidst constant proposals for regulatory change and an impending federal election, the prospects for competition in Australian electricity markets is a little unclear. On one hand, proposals like the Federal government directly intervening to build more state-owned generation (including nuclear power stations) does not bode well for a more competitive market. On the flip side, the sustained wave of new investment in storage, demand management and renewables could change the structure of the wholesale market and make it much more competitive.
There is a thread linking this wave of these new technologies, hot chips and electricity market competition. Tenuous maybe, but enough for a blog post!
Competition in the National Electricity Market
The level of competition in the wholesale electricity market in Australia has historically been less than you’d want to see in a highly competitive market.1 The ACCC have highlighted the concentration of ownership of generation assets in multiple reports, such as their 2018 and 2022 reports.
In their 2018 report (a “Major Review”), the ACCC noted they considered the level of competition in the NEM had fallen between 2011 and 2018 due to:
the NSW government selling all of its generation portfolio to existing participants, particularly highlighting the sale of Macquarie Generation to AGL, instead of breaking it up2
the QLD government restructuring its generation businesses from three entities into two.
AGL buying Loy Yang A and ENGIE closing Hazelwood.
the closure of Playford and Northern Power Stations in South Australia.
This review recommended measures to boost competition by restricting concentrated ownership of capacity, introducing greater powers to the Australian Energy Regulator (AER), creating a long-term government underwriting mechanism and implementing a form of carbon pricing.
One of the more important measures of market concentration looks at the ownership of dispatchable generation. While wind and solar generation typically reduce the wholesale electricity price when generating, they don’t often set the wholesale price. Coal, gas, hydro and batteries have greater ability to set the wholesale electricity price (see the figure below). As such, low levels of competition amongst dispatchable resources would mean that more often, the electricity price is not being set at a level you’d expect in a competitive market i.e., poor pricing outcomes for consumers.
In a separate report published at the end of last year, the Australian Energy Regulator said:
“Ownership of dispatchable generation, including flexible capacity, remains concentrated and a few large participants are often needed to meet demand. But the reliance on the largest participants has decreased across the NEM over the past two years, except in Victoria.”
So, how to improve the competitiveness of the market? Notable in the AEMO graph above is the growing purple sliver, the contribution of large-scale batteries. Battery capacity has rapidly grown in the NEM and is now setting the wholesale price up to 20% of the time in the evening peaks.
The influx of batteries into the NEM presents an opportunity to increase the level of competition in the market and deliver better outcomes to consumers. There are much lower barriers to entry for storage owners compared to building a coal, gas or nuclear plant. As the number of batteries installed continues to grow, it is easy to see how it could lead to to see a highly competitive wholesale market.
Batteries in the NEM
Large-scale batteries in the NEM need to bid into a competitive market every five minutes. Whereas the companies trading power stations of past were highly dependent on manned trading rooms, batteries are trading almost entirely through bidding software with human oversight.
This software ingests a range of information relating to price forecasts, the weather and battery constraints to try to determine and execute optimal bidding strategies.
In this regard, the NEM is highly transparent. Prices are published every five minutes, alongside demand forecasts, pre-dispatch pricing. The next day, the bids generators made are made public as well. This presents a huge amount of information for software platforms to consider when developing and refining strategies.
For batteries, getting these strategies right is extremely important. Most batteries will cycle between 1 - 1.5 times per day — making sure they are sending out power when wholesale prices are highest is key to profit maximising. Imagine selling all of your lemonade the day before a heatwave when everyone is willing to pay double!
In practice, there is a lot for a battery bidding software to contend with. Price forecasts are very difficult to get right, the impact of network constraints can be counterintuitive, and the drivers of price spikes can be hard to predict (e.g. a transmission line failure or coal generator tripping offline).
Combined with the relatively low upfront costs to develop software platforms, the battery bidding space has plenty of competition and innovation.
The hot chip cartel
In the US, there is an ongoing series of lawsuits in which plaintiffs are suing the four largest manufacturers of frozen potato products.3 The plaintiffs allege that these manufacturers have acted as a cartel, colluding to raise the prices of potato products.4
Ye olde collusion would be something like the CEOs of these four organisations meeting in some smoky underground bar to secretly agree to push their prices up together. This collusion is alleged to be much less smoky5 — the plaintiffs allege the collusion occurred through the use of third party software that recommended product pricing.
These four companies, McCain Foods, Cavendish Farms, Lamb Weston and J.R. Simplot, control about 97% of the market for frozen potato products. As such, if the four businesses were to cooperate on price fixing, they would have direct control over the price.
All four companies used a third-party platform called PotatoTrac which analyses data on a range of confidential information submitted across all four businesses. PotatoTrac will recommend pricing strategies based on the input information. The lawsuits allege that without directly sharing information with competitors, by using PotatoTrac, these four businesses were effectively colluding to set prices above competitive levels.
This issue has arisen across multiple markets, including sugar and rental properties. Across these markets, the concept is fairly simple. You and your competitors don’t share information directly — instead, you upload information into a black-box and it tells you how you should price your product in a profit-maximising way.
Across the USA, anti-trust enforcement has taken greater notice of information sharing platforms. The US Federal Trade Commission put it succinctly in a post about a case against rental price fixing:
“Price fixing by algorithm is still price fixing…
(1) you can’t use an algorithm to evade the law banning price-fixing agreements, and (2) an agreement to use shared pricing recommendations, lists, calculations, or algorithms can still be unlawful even where co-conspirators retain some pricing discretion or cheat on the agreement.”
This case is also ongoing. In December, a judge rejected the implicated software companies’ motion to dismiss the case.
The US hot chip market is obviously pretty far removed from the Australian electricity market. Competition law across the US and Australia is different. In the US, the framework is heavily dependent on lawsuits to enforce anti-trust. In Australia, collusion is illegal in Australia under the Competition and Consumer Act, which is enforced by the ACCC. However, across both jurisdictions the legal frameworks outlaw collusion.
How does this apply to batteries in the NEM?
The influence of bidding platforms has been persistently growing, applied across large-scale renewables and batteries. Similar optimisation software is proliferating behind the meter, controlling batteries, thermostats, electric vehicle charging etc.
In many respects, these algorithms are a good thing. They lower the barriers to bidding in a very complex market and can make more optimal decisions, which should in turn deliver better outcomes to consumers.
The lawsuits in the US made me think about the potential for similar issues to arise in NEM bidding software platforms. Collusion includes agreements between competitors to fix prices, control supply, rig bids and divide markets. It’s certainly possible to imagine these issues arising.
An obvious concern is the potential for concentration of bidding software platforms. If larger and larger numbers of battery operators use the same software platform, they could end up bidding in a highly correlated manner e.g., if the bidding platform has some unique strategy for predicting price spikes based on the publication of a new Currently Speaking post, then all the batteries it optimised by that bidding software will reserve capacity for when this post goes live.
Initially, this might actually be a bad thing for the controlled batteries. If they all offer their generation at the same time, they would lower the price they receive. The concern would be whether the software provider realises this and decides to bid the batteries together to push up the price. If they controlled enough batteries, this would absolutely be possible (although illegal). In the scheme of issues, market concentration of bidding software would be relatively easy to monitor (although I’m not sure it currently is).
There are more complex forms of collusion that algorithms can achieve that are much harder to monitor. In particular, self-learning algorithms can learn over time that colluding with other bidders is profit maximising. This is shown in simpler, controlled settings and likely some way off something that could apply in a much more complicated market setting like the NEM, but advances in AI will undoubtably bring about this reality eventually.
Unsurprisingly, I’m not the first person to think about this. In writing this blog post, I found an article by Jeremy Chan titled “Algorithmic collusion and Australian competition law: Trouble ahead for the National Electricity Market?”. This article outlines the two sides of the argument:
Firstly, on the pro-competition side, bidding algorithms are effective at managing volatility and predicting and responding to intraday changes. This would increase competition amongst generators and lead to lower overall prices.
On the anti-competitive front, he argues that algorithms increase the risk of collusion. He highlights autonomous algorithmic collusion — where algorithims autonomously learn to coordinate supracompetitive prices, without any actual communication. This type of collusion is especially tricky from a competition law perspective, because there is not communication between parties to enforce against.
Chan concludes, on balance, there are serious risks arising from algorithmic collusion in the NEM and goes on to discuss the potential solutions that could and should be implemented through Australian competition and electricity laws.
How much of a problem is it really?
Chan’s article is not a deep-dive on the economics of the NEM and how they are changing (it was published in a law journal, hence the focus on legal frameworks). Chan references other research indicating that markets susceptible to autonomous algorithmic collusion are ones with homogenous goods, constant demand, frequent and public pricing, lumpy entry and exit and high levels of concentration.
The NEM certainly has homogenous goods and frequent public pricing. Historically, it has had fairly constant demand, high barriers to entry and concentrated ownership of supply. But demand for electricity will be increasingly price sensitive, the scalability of renewables and storage significantly lower barriers to entry (you can put an electricity generator on your roof!) and hopefully the market will be less concentrated as coal-fired generators retire and are replaced.
In Currently Speaking style,6 it’s hard to define the problem, both now and into the future. This is likely to be a rapidly evolving space, but to mitigate against the risks of algorithmic collusion, there are some steps that can be taken:
Manage market concentration, both in the ownership of generation and the deployment of specific bidding algorithms. Firms shouldn’t be allowed to own large swathes of supply or control large swathes of supply through their algorithms.
Keep barriers to entry low. Allowing new generation to enter and exit the market allows markets to respond to prices being above competitive levels.
Review competition frameworks and if needed, look at some balanced regulatory steps. While some aspects of algorithmic collusion may be hard to capture under our current frameworks, the risks for the NEM are still unknown. Rushing to implement heavy-handed regulation would have material risks of slowly our transition away from fossil-fuels. It may be that some updates are required to our regulatory framework, but this should be balanced against the risks of putting the brakes on sorely needed investment in battery capacity.
For the avoidance of doubt, I’m not implying any company is actively participating in collusion in the NEM. Many of our readers will have a better idea of the decision making frameworks within bidding algorithms, and of competition law. Even if this isn’t you, it would be great to hear what you think!
Measuring competition in wholesale electricity markets isn’t an exact science. For example, this study from Dylan McConnell at UNSW looks at various measures of competition in the NEM.
The ACCC did try to block this sale, but it was upheld by the Australian Competition Tribunal.
The article refers to french fries, but to our predominantly Australian audience, these are of course hot chips.
Unless you’re a software person and this is smoky to you?
Specifically, my style. Alex can be more direct…
Seems to already be happening in the contingency markets, prices have been supressed for the last couple of months by batteries mostly running the same bidding platform, all of them taking a price taking strategy. While good for the consumer in the short term this could have flow on effects into the future with investment signals broken. Funnily enough most of these batteries were funded by government grants...
Great read Declan, some interesting thoughts and real concerns for bidding behaviour in the future. Keep em coming boys, always appreciated.