The Importance of Filters: Using Shapley Value Pruning to Optimize Convolutional Neural Networks

Munovas, Romanas (2023) The Importance of Filters: Using Shapley Value Pruning to Optimize Convolutional Neural Networks. Bachelor's Thesis, Artificial Intelligence.

Preview	Text bAI_2023_MunovasR.pdf Download (2MB) | Preview
	Text toestemming.pdf Restricted to Registered users only Download (126kB)

Abstract

This paper presents an analysis of the importance of filters in Convolutional Neural Networks (CNNs) and the use of Shapley value pruning to optimize these architectures. CNNs have become the industry standard for computer vision tasks, but their growing depth and parameter size are demanding more resources for training and inference. To optimize these architectures, the practice of pruning is often used to remove redundant filters or layers. The current state-of-the-art criterion for pruning in a low-data regime approximates Shapley values via Monte Carlo sampling. Computing the actual Shapley values would be optimal, however, calculating the Shapley value for a single image has a high computational complexity, which limits the application of this method. To solve this problem, this paper proposes a hybrid approach that uses a variation of Monte Carlo approximation and actual Shapley value calculation when the number of activations allows it. This approach is designed to tackle the issue of the dangers of pruning a wanted unit due to the inevitable variance created by Monte Carlo approximation in a low-sample setting. The results show that this hybrid approach significantly outperforms random pruning and slightly outperforms the exclusive use of Monte Carlo approximation. This paper also investigates the mechanisms behind the decision making of the Shapley criterion in order to gain more insight into how the scores for activations are attributed.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Lawrence, C.P.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	11 Apr 2023 09:54
Last Modified:	11 Apr 2023 09:54
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/29546

Actions (login required)

View Item