Out of Distribution Detection in a DQN using Uncertainty Quantification Methods

Sharma, Dhruvs (2023) Out of Distribution Detection in a DQN using Uncertainty Quantification Methods. Bachelor's Thesis, Artificial Intelligence.

Preview	Text bAI_2023_SharmaD.pdf Download (8MB) | Preview
	Text toestemming.pdf Restricted to Registered users only Download (125kB)

Abstract

In the current times one can see Reinforcement Learning (RL) models being applied to a variety of problems. These include robotics, industrial automation and even video games. The concerned models are not well suited for Out-Of-Distribution (OOD) inputs where they can make false predictions with high confidences. Although OOD detection is a well-researched topic in Deep Learning, OOD Detection in RL has had a lack of emphasis in terms of research until recently. In this report we take a deep Q-Network and modify it to output confidences with uncertainty using dropout and ensembles. The models are trained on the basic scenario (ID environment) from VizDoom, an API that allows one to train RL agents on preexisting game scenarios in the Doom video game. The scenario is edited to look different, say environment B, where the textures and target monster sprite are dissimilar to the training environment. After testing the models on environment B, the confidences produced show that dropout is somewhat suitable for OOD detection in the current task, while an ensemble fails to do so with higher standard deviation in the ID environment compared to the OOD environment.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Valdenegro Toro, M.A.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	09 Feb 2023 11:19
Last Modified:	09 Feb 2023 11:19
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/29236

Actions (login required)

View Item