Hiratani, naoki;
Mehta, Yash;
Lillicrap, Timothy;
Latham, Peter;
(2023)
On the Stability and Scalability of Node Perturbation Learning.
In:
Proceedings of the 36th Conference on Neural Information Processing Systems (NeurIPS 2022).
Neural Information Processing Systems (NeurIPS)
(In press).
Preview |
Text
10935_on_the_stability_and_scalabili.pdf - Accepted Version Download (2MB) | Preview |
Abstract
To survive, animals must adapt synaptic weights based on external stimuli and rewards. And they must do so using local, biologically plausible, learning rules – a highly nontrivial constraint. One possible approach is to perturb neural activity (or use intrinsic, ongoing noise to perturb it), determine whether performance increases or decreases, and use that information to adjust the weights. This algorithm – known as node perturbation – has been shown to work on simple problems, but little is known about either its stability or its scalability with respect to network size. We investigate these issues both analytically, in deep linear networks, and numerically, in deep nonlinear ones. We show analytically that in deep linear networks with one hidden layer, both learning time and performance depend very weakly on hidden layer size. However, unlike stochastic gradient descent, when there is model mismatch between the student and teacher networks, node perturbation is always unstable. The instability is triggered by weight diffusion, which eventually leads to very large weights. This instability can be suppressed by weight normalization, at the cost of bias in the learning rule. We confirm numerically that a similar instability, and to a lesser extent scalability, exist in deep nonlinear networks trained on both a motor control task and image classification tasks. Our study highlights the limitations and potential of node perturbation as a biologically plausible learning rule in the brain.
| Type: | Proceedings paper |
|---|---|
| Title: | On the Stability and Scalability of Node Perturbation Learning |
| Event: | Thirty-Sixth Conference on Neural Information Processing Systems |
| Location: | New Orleans, US |
| Dates: | 28 Nov 2022 - 9 Dec 2022 |
| Open access status: | An open access version is available from UCL Discovery |
| Publisher version: | https://nips.cc/ |
| Language: | English |
| Additional information: | This version is the version of record. For information on re-use, please refer to the publisher’s terms and conditions. |
| UCL classification: | UCL UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences UCL > Provost and Vice Provost Offices > School of Life and Medical Sciences > Faculty of Life Sciences > Gatsby Computational Neurosci Unit |
| URI: | https://discovery-pp.ucl.ac.uk/id/eprint/10166318 |
Archive Staff Only
 |
View Item |
