De Mars, Patrick;
(2022)
Reinforcement Learning and Tree Search Methods for the Unit Commitment Problem.
Doctoral thesis (Ph.D), UCL (University College London).
Preview |
Text
Thesis.pdf - Other Download (3MB) | Preview |
Abstract
The unit commitment (UC) problem, which determines operating schedules of generation units to meet demand, is a fundamental task in power systems operation. Existing UC methods using mixed-integer linear programming are not well-suited to highly stochastic systems. Approaches which more rigorously account for uncertainty could yield large reductions in operating costs by reducing spinning reserve requirements; operating power stations at higher efficiencies; and integrating greater volumes of variable renewables. A promising approach to solving the UC problem is reinforcement learning (RL), a methodology for optimal decision-making which has been used to conquer long-standing grand challenges in artificial intelligence. This thesis explores the application of RL to the UC problem and addresses challenges including robustness under uncertainty; generalisability across multiple problem instances; and scaling to larger power systems than previously studied. To tackle these issues, we develop guided tree search, a novel methodology combining model-free RL and model-based planning. The UC problem is formalised as a Markov decision process and we develop an open-source environment based on real data from Great Britain’s power system to train RL agents. In problems of up to 100 generators, guided tree search is shown to be competitive with deterministic UC methods, reducing operating costs by up to 1.4%. An advantage of RL is that the framework can be easily extended to incorporate considerations important to power systems operators such as robustness to generator failure, wind curtailment or carbon prices. When generator outages are considered, guided tree search saves over 2% in operating costs as compared with methods using conventional N-x reserve criteria. The strategies adopted by guided tree search improve our understanding of the problems studied and demonstrate that RL is a rich methodology for solving the UC problem, offering practical value to system operators through more intelligent operation of complex and uncertain power systems.
Type: | Thesis (Doctoral) |
---|---|
Qualification: | Ph.D |
Title: | Reinforcement Learning and Tree Search Methods for the Unit Commitment Problem |
Open access status: | An open access version is available from UCL Discovery |
Language: | English |
Additional information: | Copyright © The Author 2022. Original content in this thesis is licensed under the terms of the Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) Licence (https://creativecommons.org/licenses/by-nc/4.0/). Any third-party copyright material present remains the property of its respective owner(s) and is licensed under its existing terms. Access may initially be restricted at the author’s request. |
UCL classification: | UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment UCL > Provost and Vice Provost Offices > UCL BEAMS > Faculty of the Built Environment > Bartlett School Env, Energy and Resources UCL > Provost and Vice Provost Offices > UCL BEAMS UCL |
URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10157786 |
Archive Staff Only
![]() |
View Item |