Liu, L; Li, Y; Kuang, Z; Xue, J; Chen, Y; Yang, W; Liao, Q; Liu, L; Li, Y; Kuang, Z; Xue, J; Chen, Y; Yang, W; Liao, Q; Zhang, W; - view fewer (2021) Towards Impartial Multi-task Learning. In: Proceedings of the International Conference on Learning Representations (ICLR 2021). ICLR: Virtual event.

Preview

Text LiyangLiu-ICLR2021-accepted-Jan25.pdf - Accepted Version Download (41MB) | Preview

Abstract

Multi-task learning (MTL) has been widely used in representation learning. However, naively training all tasks simultaneously may lead to the partial training issue, where specific tasks are trained more adequately than others. In this paper, we propose to learn multiple tasks impartially. Specifically, for the task-shared parameters, we optimize the scaling factors via a closed-form solution, such that the aggregated gradient (sum of raw gradients weighted by the scaling factors) has equal projections onto individual tasks. For the task-specific parameters, we dynamically weigh the task losses so that all of them are kept at a comparable scale. Further, we find the above gradient balance and loss balance are complementary and thus propose a hybrid balance method to further improve the performance. Our impartial multi-task learning (IMTL) can be end-to-end trained without any heuristic hyper-parameter tuning, and is general to be applied on all kinds of losses without any distribution assumption. Moreover, our IMTL can converge to similar results even when the task losses are designed to have different scales, and thus it is scale-invariant. We extensively evaluate our IMTL on the standard MTL benchmarks including Cityscapes, NYUv2 and CelebA. It outperforms existing loss weighting methods under the same experimental settings.

Download activity - last month

Export as JSONXMLCSV

Download activity - last 12 months

Export as CSVXMLJSON

Downloads by country - last 12 months

Export as JSONXMLCSV

Archive Staff Only

View Item