Training a Physics-Based Osseointegrated Transfemoral Amputee Model with a Reduced State Observation using Deep Reinforcement Learning

Ogum, Brown (2022) Training a Physics-Based Osseointegrated Transfemoral Amputee Model with a Reduced State Observation using Deep Reinforcement Learning. Master's Thesis / Essay, Artificial Intelligence.

Preview	Text mAI_2022_OgumBN.pdf Download (10MB) | Preview
	Text toestemming.pdf Restricted to Registered users only Download (104kB)

Abstract

This project leverages a physics-based simulated environment (OpenSim) for the implementation of an intelligent control framework of an osseointegrated transfemoral amputee model, for the task of normal walking. A Deep Reinforcement Learning (DRL) algorithm - Proximal Policy Optimization (PPO) (with imitation learning) is used to optimize a policy for the walking task. Training the model in a simulation gives us access to additional muscle information which are not readily observable in the real world. The main aim of this research is to observe if a transfemoral amputee model can be trained to walk using Deep Reinforcement Learning while using a reduced number of state observers. To this end, a transfemoral amputee agent is trained to walk using the complete observation state - which contains kinematic data of the agent, including the force, length, and velocity of the agent’s muscles. The agent is also trained by observing a reduced state. This reduced state only contains data that can readily be obtained in the real world with devices such as Inertia Measurement Units (IMUs) and rotary encoders. Hence, the muscle information is not included in this state representation. The effects that the lack of muscle forces and fiber length and velocity information have on the generation of gait are observed using three symmetry measures - RMSE, symmetry angle, and trend symmetry. Several Deep Neural Network architectures were trained in a supervised manner to predict the missing muscle information of the reduced state of the prosthesis model and their results were appraised using 5-fold cross-validation. It was observed that a fully-connected feed-forward neural network with 3 hidden layers had the best performance on the prediction task. Lastly, empirical results using an observation state that was augmented with the predicted muscle information showed that the transfemoral amputee model can be trained to walk using this framework with comparable rewards and symmetry to using the complete observation.

Item Type:	Thesis (Master's Thesis / Essay)
Supervisor name:	Carloni, R. and Schomaker, L.R.B.
Degree programme:	Artificial Intelligence
Thesis type:	Master's Thesis / Essay
Language:	English
Date Deposited:	28 Sep 2022 08:51
Last Modified:	10 Jan 2024 13:30
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/28772

Actions (login required)

View Item