Predicting future energy demands in a complex market

Inaccurate forecasting carries high stakes. Underestimating demand could lead to costly power shortages and outages. Overestimating leads to billions in wasted investment, ultimately raising costs for consumers. The need for precise forecasting is now greater than ever.

King’s mathematicians have been developing a completely new kind of statistical modelling tool for this purpose, which can be tailored to a whole raft of different scenarios and variables. This enables users to understand the many uncertainties around long-term forecasts and develop more accurate predictions.

The road to Net Zero emissions

To achieve the UK’s aim of Net Zero emissions by 2050, the government has set an ambitious target to build, connect, and operate a near-decarbonised electricity grid by 2030, while maintaining security of supply.

To help drive this forward this change, the Low Carbon Contracts Company (LCCC) was established to run the Contracts for Difference (CfD) scheme. The scheme incentivises investment in renewables by providing developers with protection from volatile wholesale prices, and consumers from paying increased costs when electricity prices are high.

It has resulted in 570 contracts to date covering generation technologies across wind, solar, nuclear, biomass, tidal, and more.

To be able to accurately cost the CfD scheme for investors and consumers, the LCCC require exceptionally accurate electricity forecasts. Yet legacy models have not kept pace with the developing electricity market, given the complexity of the data and plethora of different variables have been very slow to update and deploy.

Future proofing electricity forecasts

The LCCC reached out to King’s College London to propose a new approach to modelling national demand, in a way that allows for describing uncertainties around long-term forecasts, while capturing regular statistical patterns on different time scales.

Chiwi Nwokenna, Head of Analytics at LCCC, said “continuously improving the quality of our forecasts is vital - it saves money for consumers and producers.

"But we were grappling with this challenge of how we evolve forecasting models that we've had as an industry for a long time to account for the changing dynamics of the future.

“We approached King’s to see how we could stretch the current thinking and build a forecasting model that was more reflective of future electricity demand.”

Working with the LCCC, statisticians from the Department of Mathematics were able to build a prototype modelling tool, able to enabling flexibly to adapt to any scenario or variable, and user friendly without the need for expert knowledge.

A new, intuitive model for accurate forecasting

Project lead, Professor Teemu Pennanen explains, “What’s different about our approach is that instead of starting with existing statistical models and mathematical structures as the basis, we started with the problem we were trying to solve.

“We know the electricity market faces great uncertainties, so modelling needs to account and reflect that. What makes our model different, is that we built in a high degree of uncertainty of demand, while capturing daily, weekly and annual seasonalities and other regularities that can be observed in the historical data going back decades.”

The other innovative aspect is the ease with which users can input their own data, views or predictions, including those based on expert knowledge, such as how the rapid rise of AI and EVs will impact demand.

The model has other benefits, as Teemu explains, "It’s quicker to compute, but also has an intuitive structure and is more transparent and clearer for non-expert statisticians.

“This is very important when it comes to calibrating probabilistic models - if you don’t understand what the model is doing, it’s very risky for the user in terms of creating forecasting they can interrogate and make sense of.”

It also provides a whole range of different scenarios - the team input expert views into the model and were able to draw thousands of scenarios, with different probabilities, variabilities, criteria, which could then be interrogated and refined by experts. This process allows users to arrive at a much more concrete and tailored response to their specific requirements.

For example, the model can share forecasts at half hourly frequency, taking into account variability due to time of day, day of the week, and the annual variation, including holidays. Working these features into the model allows for much more detailed and comprehensive analysis - structures that traditional statistical generic models don’t have.

Chiwi said, “working with academics has been important to make sure that we are connected to best in class thinking, understanding what the best ideas are out there to innovate, challenge the thinking and ideas that we have, and ensure our internal forecasting models are fit for purpose.

“There are many common goals and challenges. Increasingly you're seeing more organisations sharing data, sharing ideas, and trying to solve problems collectively. We found it a very fruitful exchange. We have data and problems. They have research and expertise. And we can come together and solve problems successfully.”

The future of electricity forecasting?

Whilst the prototype was developed as a bespoke tool for the LCCC, the underlying approach could be adapted and deployed across any country or market by anyone working in energy.

It is particularly useful for risk management, as Teemu explains, “we are able to better quantify the uncertainty and describe the risks, leading to a better understanding of the market price of electricity, which is important in finance and financial contracts.

"After all, uncertainties drive the price, so we want to fairly price that in, for consumers and companies alike.”

The team are also exploring how to build similarly sophisticated models to support forecasting on the supply side. This includes working with companies on the way renewable energy generation and storage is shaping this.

Teemu said, “renewable energies require energy storage solutions and we are aiming to develop models that can optimise the management of battery storage, including drawing on expert views.”

“By creating a model that factors in the uncertainties of renewable energy generation with recommendations for optimising battery storage, it will help companies develop better business strategies.

"There will be more flexibility on when to sell, more room for optimisation of their revenue and, ultimately it will empower them to come up with the optimal plan for producing, buying and selling.”