On the Scalability of Deep Inverse Reinforcement Learning in High-Dimensional State Spaces

Maathuis, Hendrik, H (2020) On the Scalability of Deep Inverse Reinforcement Learning in High-Dimensional State Spaces. Master's Thesis / Essay, Artificial Intelligence.

Preview	Text ThesisFinalHenry.pdf Download (1MB) | Preview
	Text toestemming.pdf Restricted to Registered users only Download (113kB)

Abstract

Inferring intentions behind observed behaviours is arguably an important aspect when learning new tasks. Inverse Reinforcement Learning (IRL) is a relatively small and recent area which studies techniques which allow for deriving reward functions on the basis of observed behaviour. Such a reward function can be seen as a blueprint and stipulates how an agent should behave and what it should achieve. Once a reward function is in place, it is possible to learn a policy. Normally in a Reinforcement Learning task, the reward function is handcrafted by a human. IRL therefore requires an extra abstraction to first derive the reward function before a policy can be learnt. One of the big problems in IRL is that the algorithms do not scale well to high-dimensional environments due to overfitting of the reward function during the reconstruction process. As such, this thesis contributes to the field of IRL by accessing the state-of-the-art Adversarial Inverse Reinforcement Learning (AIRL) algorithm in combination with dimensionality reduction techniques. Different autoencoders are considered including one that forms a discrete latent distribution over the data. The algorithms are evaluated on several environments: Catcher, Pong and Freeway. The results indicate that autoencoders could be useful in cases where the state space is relatively complex. Although, one should be careful when introducing autoencoders as they can also thwart the process of learning reward functions.

Item Type:	Thesis (Master's Thesis / Essay)
Supervisor name:	Wiering, M.A. and Schomaker, L.R.B.
Degree programme:	Artificial Intelligence
Thesis type:	Master's Thesis / Essay
Language:	English
Date Deposited:	01 Oct 2020 09:05
Last Modified:	01 Oct 2020 09:05
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/23460

Actions (login required)

View Item