Transferability of Vision Transformers: The key to success on small art classification datasets?

Tonkes, V. (2022) Transferability of Vision Transformers: The key to success on small art classification datasets? Bachelor's Thesis, Artificial Intelligence.

Preview	Text bachelors_thesis_vincent_tonkes.pdf Download (3MB) | Preview
	Text Toestemming.pdf Restricted to Registered users only Download (121kB)

Abstract

Convolutional Neural Networks (CNNs) have become the de facto standard in Computer Vision, and played a major role in the advancement of this field. In recent years, however, Vision-Transformer-based models (VTs) have been outperforming CNNs across the board. While this is exciting, CNNs still have an advantage on small datasets, due to their so-called image-specific inductive bias. Recent work suggests that transfer learning methods allow VTs to compete with CNNs on some of these small datasets. This thesis investigates whether that also holds true for art classification problems within the digital humanities. To this end, it compares popular VTs and CNNs in terms of their off-the-shelf and fine-tuning transferability, when going from ImageNet1K to target tasks provided by the Rijksmuseum Challenge dataset. The results show that VTs possess superior off-the-shelf feature extraction capabilities here, and that in general, transfer learning allows VTs to become an interesting alternative to consider within the digital humanities.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Sabatelli, M.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	10 Aug 2022 06:18
Last Modified:	10 Aug 2022 06:18
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/28318

Actions (login required)

View Item