Extracting Concepts From Neural Networks Using Conceptor-based Clustering

Peters, Joris (2024) Extracting Concepts From Neural Networks Using Conceptor-based Clustering. Bachelor's Thesis, Artificial Intelligence.

Preview	Text AI2024PetersJ.pdf Download (1MB) | Preview
	Text Toestemming.pdf Restricted to Registered users only Download (124kB)

Abstract

Conceptors are versatile neuro-symbolic formalizations of concepts as they arise in neural networks, with promising results on supervised tasks. However, the use of conceptors in unsupervised settings remains largely unexplored. Meanwhile, previous brain science and AI research used clustering to extract concepts from neural representations. This study combines conceptor-based representations with clustering methods for the unsupervised extraction of human- meaningful and coherent concepts from the activations of neural networks. Concretely, experiments are conducted on the responses of an Echo State Network (ESN), a type of recurrent neural network, to phoneme utterances from the TIMIT Acoustic-Phonetic Continuous Speech Corpus. In preparation, conceptor-based classification was demonstrated, and ESN hyperparameters were tuned. Then, two clustering methods, generalized centroid-based hard clustering and hierarchical agglomerative clustering, are adapted to work with conceptors and extract concepts from the ESN’s responses. The resulting concepts and concept hierarchies were significantly human-meaningful, resembling established phonetic categories, and coherent. Conceptor-based clustering, although in its infancy, represents a promising approach to unsupervised concept extraction and forming conceptors without supervision. Applications in neuro-symbolic computational creativity, brain sciences, time-series clustering, and neural network explainability are suggested.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Jaeger, H. and Pourcel, G. A.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	29 Oct 2024 06:46
Last Modified:	29 Oct 2024 06:46
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/34349

Actions (login required)

View Item