A computational model for continual learning and synaptic consolidation
Options
BORIS DOI
Abstract
How humans are able to learn and memorize is a long-standing question in science. Much progress has been achieved in recent decades to answer this question but the are still many open problems. One of these problems refers to the human ability to learn several tasks in sequence without forgetting.
In neuronal networks learning can interfere with pre-existing memories when the network is engaged in continual learning. The interference is particularly pronounced if, for instance, similar sensory stimuli require different responses depending on the context. Unlike in humans, this can lead to a memory loss termed catastrophic forgetting. To avoid interference and its fatal consequences, only a subset of synaptic weights should be consolidated. In this work we propose as computational model which performs selective consolidation by incorporating the synaptic tagging and capture hypothesis. This hypothesis, well grounded by experimental evidences, claims that synaptic consolidation requires both a synaptic-specific tag and diffusible plasticity-related proteins. We show that synaptic tagging and capture can be modeled by two classes of synaptic processes acting on different time scales. The two classes, characterized whether protein synthesis is required, are represented in our model by two synaptic components interacting with each other.
With our approach we demonstrate that synaptic consolidation can not only diminishes the problem of catastrophic forgetting during continual learning but also enables fast learning through strongly changing synaptic strengths during the early phase of long-term potentiation. The model reproduces various experimental observations on synaptic tagging and cross-tagging. It also explains why learning in psychophysical experiments is hampered when different types of stimuli are randomly intermixed.
In neuronal networks learning can interfere with pre-existing memories when the network is engaged in continual learning. The interference is particularly pronounced if, for instance, similar sensory stimuli require different responses depending on the context. Unlike in humans, this can lead to a memory loss termed catastrophic forgetting. To avoid interference and its fatal consequences, only a subset of synaptic weights should be consolidated. In this work we propose as computational model which performs selective consolidation by incorporating the synaptic tagging and capture hypothesis. This hypothesis, well grounded by experimental evidences, claims that synaptic consolidation requires both a synaptic-specific tag and diffusible plasticity-related proteins. We show that synaptic tagging and capture can be modeled by two classes of synaptic processes acting on different time scales. The two classes, characterized whether protein synthesis is required, are represented in our model by two synaptic components interacting with each other.
With our approach we demonstrate that synaptic consolidation can not only diminishes the problem of catastrophic forgetting during continual learning but also enables fast learning through strongly changing synaptic strengths during the early phase of long-term potentiation. The model reproduces various experimental observations on synaptic tagging and cross-tagging. It also explains why learning in psychophysical experiments is hampered when different types of stimuli are randomly intermixed.
Date of Publication
2020
Year of graduation
2020
Theses Type
dissertation
Subject(s)
Language(s)
en
Author(s)
Leimer, Pascal |
Faculty/Graduate School
Institute
Access(Rights)
open.access
Primary OA Publication
true