Bidding algorithms
Don't miss the forest for the trees.
Alex and I are hanging around All Energy. If you see us and you wanna chat, please say gday. Just don’t mention all of my typos, I wrote this in a hurry!
It's been a bad few months to be an algorithm! With the exception of a special subset of people, algorithms are fighting an uphill battle to get some love. Their PR job was made harder by the Nelson Panel’s review, an AEMC staff paper and this pesky internet rag…
The focus has been on bidding algorithms and their impacts of electricity markets. Bidding algorithms (also referred to as autobidders) play a key role in determining how batteries participate in the NEM. However, the current focus has painted a mostly negative, concerning picture of what the growing use of bidding algorithms means, even to the point of suggesting constraint and regulation of bidding algorithms.
The risk is that this narrative overwhelms what is actually an overwhelmingly positive story - the story of how batteries can (are already are) addressing one of the most important historical challenges of our market, wholesale market concentration.1
Electricity traders and autobidders
Electricity systems have always been very dynamic. Supply and demand are matched in real-time and so, for the history of electricity systems, large-scale generators have generally been required to turn generators up and down to follow fluctuations in demand.
Before the NEM started, the state electricity commissions aimed for efficient dispatch by using the cheapest generators first, before moving to more expensive generation when demand was higher.2
With the introduction of a wholesale market for electricity, the dynamic balancing of supply and demand was no longer decided by one central entity. The operators of power plants could make their own assessments of how they should run their generator and the price they would bid them into the market. The traders for that generator make bids for generators and submit them electronically to AEMO. Through a competitive wholesale market, these power plant operators were incentivised to find strategies to beat their competitors.
These traders developed deep insights into how to position their assets to compete for market share. However, having a team of traders monitoring the market and determining optimal strategies was expensive. This was the cost of entry to the electricity market when it was set up and one (of many) reasons you didn’t see many small players setting up shop.
This started to change with large-scale renewables. While it’s not cheap to build a solar- or wind-farm, the scalability of renewable generators meant some new players could build, own and operate generators. If the owners of renewable generators typically didn’t already have a trading floor, they preferred not to have to set one up. Instead, they just wanted to be paid when the sun was shining, wind blowing etc. As a solution, they relied more heavily on bidding software than the traditional generators.
Renewable generators had simple bidding strategies, so these software programs didn’t need to replicate a trading team with a sophisticated strategy. Rather, bidding software offered a much more cost-efficient approach to submit bids.
The developers of large-scale batteries, which are still relatively new in the NEM, similarly sought cost-effective approaches for participating in market. However, unlike wind and solar, they needed much more sophisticated strategies. Bidding batteries into the NEM in an optimal way is exceedingly difficult. It combines the extreme levels of flexibility (batteries can flip from charging to discharging almost instantly), operating constraints (cycling limits, etc) and the variability of the wholesale market (as well as the related FCAS markets). Solving this challenge is where autobidders entered the NEM in a big way.
What are autobidders and what do they do?
In the context of the NEM, an autobidder is a broad term for a software package that deals with parts of the complexity of constructing a bid. It can do any combination of the following:
Monitor market conditions, estimate the times when electricity is going to be most valuable and form a best attempt at an optimal bidding strategy. The strategy adapts in real-time as market conditions change.
Package this bidding strategy into actual bids and rebids that are submitted to AEMO, ingesting and actioning dispatch instructions.
Fit a BESS into a broader portfolio of assets working together.
Manage constraints, including any obligations to hold some power in reserve.
Every large-scale battery uses some form of bidding platform. A large battery in the NEM without one would either be hopelessly outcompeted or require a large dedicated team of traders. Bidding platforms are so important to BESS operators that there is an observable competition amongst providers. Some of the well known autobidders include Tesla’s Autobidder and Fluence’s Mosaic, and newer entrants like Optigrid and Hachicko.
Because it's a market for software, there are relatively low barriers to entry and the size of the market is huge. On average, a large-scale battery in the NEM earned $150,000/MW in 2024. This means an average 100MW battery pulled in $15M in 2024. In an increasingly competitive market, a good bidding platform could be the difference between a battery that pays itself back or one that doesn’t.3
What are the concerns with autobidders?
If you would like to understand the concerns about autobidders in detail, you should have a read of the papers written by the AEMC and the Nelson Panel. In short, the primary concerns relate to:
The level and frequency of rebidding, and how this may impact the effectiveness of the market.
The potential for collusion. In short, where market participants agree to push up the price together. This can happen explicitly (i.e. openly agree to collude) or tacitly (where collusion occurs without open agreement).
The AEMC focussed in particular on the risks of collusion with the growing use of AI and bidding algorithms. The Nelson Panel’s review agreed with the AEMC’s recommendation, suggesting:4
“Market bodies and the ACCC should collaborate to develop rule changes and update regulatory responses to ensure that rebidding and algorithmic bidding contribute to efficient price formation and do not distort forecasting and price discovery.”
How real are these concerns though?
Rebidding
Market participants submit their wholesale market bids at the start of the day. Over the course of the day, they can rebid i.e., change these bids. There are good reasons for doing this. There is generally more information available (such as weather updates, a generator tripping offline, demand forecast updates etc.) and this changes what participants want to do.
The amount of rebidding has significantly increased over the past five years. The Nelson review highlighted this in the below graph.
The draft paper did not do a great job explaining why this is a problem though.5 Instead, it just suggested that this could make checking the rebidding rules compliance more difficult, which is undoubtably true. However, it’s also true that batteries just naturally need to rebid more often. They have limited capacity and are highly focussed on optimising this capacity. If market conditions change, it will proportionally lead to rebids on a greater scale than you would see with other types of generation.
The Nelson review does reference this paper from Abhijith Prakash, Anna Bruce and Iain Macgill from UNSW. Abhi, Anna and Iain do a much more thorough job discussing the potential inefficiencies arising from higher levels of rebidding.
However, this also isn’t just a battery problems. There has also been a huge growth in the rebids from PV, wind, demand response/virtual power plants as well as batteries. In 2021, they were collectively responsible for over 40% of the rebids, and all of them are likely heavily reliant on bidding platforms.
Noting the points raised in Abhi’s paper (which is a great read), there is a case to go the other way. The generation mix is changing. Clunky, old, stable coal units are being replaced with variable generation and highly controllable storage. It makes sense that there will be more bidding and rebidding. Restrictions of rebidding risk curtailing the flexibility in the fleet of batteries. Conversely, reducing regulation around rebidding could allow for a more optimal use of the fleet of BESS.
Does this all suggest rebidding and rebidding rules need changing? I don’t think a compelling case has been made either way. But if the conversation is going to gravitate to restricting rebidding, we should be clear on the costs of doing so.
Tacit collusion
The AEMC’s paper focussed heavily on tacit collusion. However, on reading the paper and the examples put forward, I think it’s clear that suggesting this is something that could occur in the NEM is a long bow to draw. The examples used in the paper look nothing like the NEM, and nothing that I’ve seen in any academia suggest the NEM would be susceptible to algorithmic collusion. This does not mean it’s impossible, but it also means there’s good reasons to not get overexcited/nervous.
For tacit algorithmic collusion to occur, algorithms designed to maximize profits need to learn to recognize and respond to competitors’ pricing strategies. The result would be a stable, collusive outcome where prices remain artificially high.
While we should definitely never say never, here’s some reasons why it might not be worth getting too worked up over it.
The battery owner still holds the pen. A market where tacit collusion emerges would likely rely on algorithms having absolute control over the bidding strategy. However, in practice, even while a battery owner might outsource the vast majority of the complexity to an autobidder, the owner will still be able to impose it’s own constraints and strategy. This makes it more difficult for batteries to reinforce the same strategies.
The BESS owners don’t have the same profit maximising objectives. Related to the above, while each battery owner may be using algorithms and software to support operation of the BESS, each BESS is going to have a different operational strategy. Those owned by gentailers form part of a larger portfolio of assets managing an overall position gentailing. Other batteries are set up under long-term offtakes which may have it’s own operational constraints or directives, while others are providing network services etc. The end result is bidding strategies that aren’t directly aligned in terms of how they profit maximise, which makes it more difficult to align strategy.
There are also commercial opportunities associated with taking a different strategy. Tacit collusion only really works when the vast majority of the players with market power play ball. However, at the same time, there is a commercial opportunity to take alternative strategies as the amount of storage grows. If most of the BESS were controlled by a similar strategy, it would dampen the price spikes those BESS respond to. By breaking from the group and taking a different strategy, a BESS would both be greater commercial opportunities and undermining the likelihood for any collusion to emerge.
The software providers have strong incentives not to. The parties providing the bidding software do not want to create an algorithm that colludes. This is illegal and at best, offers marginal gains to their customers. This isn’t a guarantee it won’t happen, but the incentive for the companies developing bidding algorithms to want this to happen is extremely low.
Tacit collusion might well be impossible in the NEM, even with very advanced AI. Taking a step back and looking at where tacit collusion has emerged, it still requires a fairly specific market structure. The US potato cartel example I looked at in my last post on this topic was looking at a concentrated market using a single platform. The NEM is highly complex and highly regulated and is a much less fertile ground for these sorts of behaviours. The AEMC paper talks about the levels of information available in the NEM as a contributor to the risks of collusion. However, the complexity of the market and the large number of participants suggest to me that it’s possible that under no circumstance is this even possible. This is also before we take into account the rapidly growing amount of distributed storage and the impending growth of vehicle-to-grid.
The forest we can’t miss!
The biggest reason we shouldn’t overreact to the growth of autobidders is because we’ll forget what they’re actually doing!
As covered in other posts, one of the most critical things for the effective functioning of the NEM is competition (unless you ask the Electricity Networks Australia…). Since its inception, there have been large players that have maintained dominant positions. The graph below points to the historically concentrated markets for generating electricity (taken from the Nelson review). Sitting about the green dotted line suggest a market that is highly concentrated.
The very exciting outcome that batteries and autobidder can deliver is a more competitive market, where the positions of these larger players are chipped away. Already, we’re seeing a huge influx of capacity owned by non-legacy operators. The diversification of the ownership of generating capacity is a fantastically good thing for the NEM and should be strongly encouraged.
This is all before we start to consider the impact of smaller household and business batteries. Trying to accurately forecast the number of small scale batteries in the NEM in 2050 is as fruitless as arguing with retired petroleum engineers on LinkedIn (unless you ask Alex…), but there could easily be 60GW of small-scale batteries including EVs owned not by multibillion dollar companies (unless you ask the Electricity Networks Australia…), but by households. Unlocking the value of all this distributed capacity will require algorithms.
Conclusion
Undoutably there has been a rapid growth in the use of autobidders and the Nelson review and AEMC and other are right to ask questions about what this could mean for the NEM. However, both also made a comment along the lines of:
“Restricting the use of AI or pricing algorithms in totality is also unattractive, given its potential benefits”
This is a massive understatement and to me, reflects the imbalance in this conversation. The “potential benefits” is lowering the barriers to entry for a wave of utility-scale and distributed batteries that will upend the wholesale market and break up market concentration.
The AEMC and Nelson review both recommend regulators look into this further. In my opinion, this is worthwhile, but it should probably start with a far greater understanding of the current landscape for algorithmic bidding. There should also be much more emphasis on the very real risks of overregulating.
As risk of overstating the impact of my blogging, I’m aware that I did also contribute to the discussion of these questions.
Someone who was there can correct me, but I’m confident this was orchestrated by a bunch of engineers in short-sleeve shirts and pens in their front pocket.
For the sake of the energy transition, it’s important that batteries can make money - we want people to keep building them!
National Electricity Market wholesale market settings review - draft report p. 107.
The paper has a page reference to the AEMC submission which refers to concerns about levels of rebidding, but the page reference seems to be incorrect?
Nicely put, Declan! My angle on the topic is that the NEM needs to support more research and scrutiny and better integration of autobidders and storage, in general, such as:
1. Make all relevant data on bidding available (eg state of charge) and monitor how well bidding and price discovery is working as the number of autobidders increases
2. Pre-process daily bidding data files to remove all the irrelevant zeros to reduce the size of bid files AEMO publishes every day
3. Develop a better form for submitting bids (two-dimensional flat file is inefficient) and review rule requirements around rebidding. For example, remove the requirement to set ten price bands at 12.30pm each day for the next NEM trading day as it creates price discovery friction that has the opposite effect to the one intended (to curb market power).
Under item 1, I note that AEMO is publishing regional state of charge but not the sum of the min and max capacity of that storage, which is a bit hopeless because the point of this information is to help traders (including autobidders) assess how close to full or empty the region's batteries are. I am hoping AEMO will fix this oversight.
12% of rebidding is steam generation. Given the small number of steam generators, I don't think 13% by batteries is "big" given their flexibility or 15% wind, given their numbers & dependence on weather.
I don't think NEM or government responded to Sonwy Hydro's manipulation of its supply to maximise prices (despite being givt owned) eg leaving gas plants off to drive prices well above their profitable level. Actually Martisson gave them an extra gas plant in the Hunter which boosts their market power.
It took years to respond to manipulative 5 minute bids under 30 minute price settlement (a stupid system seemingly designed to reduce the volume of calculations, without awareness of the risks)
Better to investigate the bidding practices of coal & gas generators eg huge prices for the last 1MW.