Javascript must be enabled for the correct page display

Exploring the Role of Representational Drift in Overcoming Plasticity Loss in Artificial Neural Networks

Comin, Arturo (2024) Exploring the Role of Representational Drift in Overcoming Plasticity Loss in Artificial Neural Networks. Master's Research Project 2, Biology.

[img]
Preview
Text
arturocomins6068553mastersthesis2.pdf

Download (1MB) | Preview

Abstract

Biological neural circuits can maintain stable behaviour while internal representations continue to change over time, a phenomenon known as representational drift. Artificial neural networks trained in continual learning settings often suffer from catastrophic forgetting and loss of plasticity, yet how standard continual learning strategies constrain internal representation change remains poorly characterised. This project quantifies representational change under regularization-based, replay-based, and architecture- based continual learning on a 50-context permuted-MNIST benchmark, comparing naive fine-tuning, synaptic intelligence (SI), experience replay (ER), and context-dependent gating (XdG) paired with SI, under both Adam and SGD and under task-incremental and domain-incremental readouts. Change is measured with a multi-metric pipeline (linear CKA, orthogonal Procrustes distance, and parameter and representational cosine distances) and related to forgetting and learnability on new tasks. SI suppresses geometric change more than ER when accuracy is matched. Under SGD, forgetting is higher, and the accompanying representational change is correspondingly larger. Accuracy-based loss of plasticity remains small at this horizon, except for SGD with replay. Deliberately inducing internal variability rarely increased measured drift without harming retention, although mild dropout revived silent hidden units.

Item Type: Thesis (Master's Research Project 2)
Supervisor name: Ven, G.M. van de and Havekes, R.
Degree programme: Biology
Thesis type: Master's Research Project 2
Language: English
Date Deposited: 02 Sep 2026 09:54
Last Modified: 02 Sep 2026 09:54
URI: https://fse.studenttheses.ub.rug.nl/id/eprint/37534

Actions (login required)

View Item View Item