Exploring Conceptor Regularization for Incremental Learning of Segmentation Tasks

Langemheen, Hogir (Kyle) van de (2022) Exploring Conceptor Regularization for Incremental Learning of Segmentation Tasks. Master's Thesis / Essay, Artificial Intelligence.

Preview	Text Master_Thesis_Conceptors_Kyle_van_de_Langemheen.pdf Download (29MB) | Preview
	Text toestemming.pdf Restricted to Registered users only Download (122kB)

Abstract

Semantic segmentation is one of many tasks in computer vision where convolutional neural networks (CNNs) are state of the art. However, catastrophic interference prevents feedforward networks from learning a new task in an incremental manner without performance loss on previously learnt tasks. To overcome this, a regularization technique named conceptor-based pseudo-rehearsal (CPR) has recently been adapted for CNNs. Incremental training with CPR has been shown to yield results close to traditional training with fully connected and partially convolutional networks, but has yet to reach similar performance with fully convolutional networks. Segmentation, as opposed to classification, is often solved with fully convolutional networks. To allow for CPR to be used in segmentation, it needs to be adapted to fully convolutional networks. We apply CPR to an incremental adaptation of the Cityscapes dataset, and find that the regularization term is initially too large and becomes numerically unstable. A novel hierarchical conceptor, intended to regulate entire convolutional filters rather than individual weights alone, is introduced as an extension to CPR. These hierarchical conceptors allow for CPR to be used on networks with many weights and filters per layer, but CPR remains insufficient for solving the segmentation task incrementally. We show that, to combat the observed numerical instability, changes to the loss and optimization scheme are required.

Item Type:	Thesis (Master's Thesis / Essay)
Supervisor name:	Jaeger, H. and Mohades Kasaei, S.H.
Degree programme:	Artificial Intelligence
Thesis type:	Master's Thesis / Essay
Language:	English
Date Deposited:	06 Apr 2023 12:20
Last Modified:	06 Apr 2023 12:20
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/29522

Actions (login required)

View Item