Henry, L; Genest, B; Drewery, A; (2023) Reinforcement Planning for Effective ε-Optimal Policies in Dense Time with Discontinuities. In: 43rd IARCS Annual Conference on Foundations of Software Technology and Theoretical Computer Science (FSTTCS 2023). (pp. 13:1-13:18). Schloss Dagstuhl - Leibniz-Zentrum für Informatik: Dagstuhl, Germany.

Preview

Text LIPIcs.FSTTCS.2023.13.pdf - Published Version Download (916kB) | Preview

Abstract

Lately, the model of (Decision) Stochastic Timed Automata (DSTA) has been proposed, to model those Cyber Physical Systems displaying dense time (physical part), discrete actions and discontinuities such as timeouts (cyber part). The state of the art results on controlling DSTAs are however not ideal: in the case of infinite horizon, optimal controllers do not exist, while for timed bounded behaviors, we do not know how to build such controllers, even ?-optimal ones. In this paper, we develop a theory of Reinforcement Planning in the setting of DSTAs, for discounted infinite horizon objectives. We show that optimal controllers do exist in general. Further, for DSTAs with 1 clock (which already generalize Continuous Time MDPs with e.g. timeouts), we provide an effective procedure to compute ?-optimal controllers. It is worth noting that we do not rely on the discretization of the time space, but consider symbolic representations instead. Evaluation on a DSTA shows that this method can be more efficient. Last, we show on a counterexample that this is the furthest this construction can go, as it cannot be extended to 2 or more clocks.

Download activity - last month

Export as XMLJSONCSV

Download activity - last 12 months

Export as JSONCSVXML

Downloads by country - last 12 months

Export as XMLJSONCSV

Archive Staff Only

View Item