Reward Machines: Effects of Noisy Labelling Functions in Complex Grid Environments in Reinforcement Learning

Erdelez, Andro (2024) Reward Machines: Effects of Noisy Labelling Functions in Complex Grid Environments in Reinforcement Learning. Bachelor's Thesis, Artificial Intelligence.

Preview	Text finalversionAE.pdf Download (1MB) | Preview
	Text toestemming_ Andro Erdelez _ degree programme_ Artificial Intelligence.pdf Restricted to Registered users only Download (145kB)

Abstract

Reinforcement learning (RL) commonly treats reward functions as black boxes, requiring extensive interaction with the environment for discovering rewards. Recently, a novel approach called reward machines (RM) has been introduced. These are finite state machines that make reward functions explicit, exploiting their internal structure. Innovative algorithms that enhance RL efficiency using RMs have emerged, demonstrating better performance than standard Q-learning. These RM algorithms rely heavily on a labeling function to determine the next state in an RM. However, it is assumed that the event detectors, defining the labeling function, operate perfectly, which is rarely the case in real-world scenarios. Moreover, these algorithms are tested with RMs that do not provide much free choice to the RL agent. This thesis addresses two questions: How does noise at different levels affect the performance of the RM algorithms when introduced to the labeling function? and How do these RM algorithms perform in a complex grid environment with significant free choice for the RL agent? To answer this, different levels of noise are introduced into the labeling function and the RM algorithms are evaluated in a new complex grid environment. Our results show the extent to which noise affects the RM algorithms’ performance, robustness and adaptability under more realistic conditions. These contributions enhance the understanding of using RMs in RL and identify potential problems.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Grossi, D.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	17 Jul 2024 14:09
Last Modified:	17 Jul 2024 14:09
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/33501

Actions (login required)

View Item