Cella, L; Lounici, K; Pacreau, G; Pontil, M; (2023) Multi-task Representation Learning with Stochastic Linear Bandits. In: Proceedings of Machine Learning Research (PMLR). (pp. pp. 4822-4847). MLResearchPress

Preview

PDF cella23a.pdf - Published Version Download (1MB) | Preview

Abstract

We study the problem of transfer-learning in the setting of stochastic linear contextual bandit tasks. We consider that a low dimensional linear representation is shared across the tasks, and study the benefit of learning the tasks jointly. Following recent results to design Lasso stochastic bandit policies, we propose an efficient greedy policy based on trace norm regularization. It implicitly learns a low dimensional representation by encouraging the matrix formed by the task regression vectors to be of low rank. Unlike previous work in the literature, our policy does not need to know the rank of the underlying matrix, nor does it requires the covariance of the arms distribution to be invertible. We derive an upper bound on the multi-task regret of our policy, which is, up to logarithmic factors, of order T √rN + √rNTd, where T is the number of tasks, r the rank, d the number of variables and N the number of rounds per task. We show the benefit of our strategy over an independent task learning baseline, which has a worse regret of order T √dN. We also argue that our policy is minimax optimal and, when T ≥ d, has a multi-task regret which is comparable to the regret of an oracle policy which knows the true underlying representation.

Download activity - last month

Export as JSONCSVXML

Download activity - last 12 months

Export as XMLCSVJSON

Downloads by country - last 12 months

Export as CSVXMLJSON

Archive Staff Only

View Item