Bikowski, Yara (2025) Detection of Confusion in Facial Expressions of Older Adults during Conversation with a Social Robot. Master's Thesis / Essay, Artificial Intelligence.
|
Text
mAI2025BikowskiY.pdf Download (5MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
Toestemming.pdf Restricted to Registered users only Download (181kB) |
Abstract
Social robots are increasingly used in elderly care. However, misunderstandings and confusion often occur during human-robot conversations. This study aims to recognize confusion during conversa tions between older adults and social robots with a focus on facial expressions. We collected data by having participants play a word game with the social robot. Facial action units were extracted from the data and an LSTM model was trained to recognize confusion. A second model was trained by using transfer learning on the ResEmoteNet model. Neither of these models were able to accurately differentiate confusion and non-confusion when tested on data from new participants. The LSTM model obtained an accuracy of 57% while the ResEmoteNet model obtained an accuracy of 53% on balanced data. This indicates that they were unable to generalize to new faces. These findings suggest that models trained on our newly collected dataset are not able to recognize confusion solely based on the facial expressions of older adults. However, earlier research has obtained higher accuracy for this task with datasets of younger adults. This suggests that our inability to detect confusion is a limi tation of the new dataset rather than the task itself. Future work should explore multimodal confusion recognition using speech and gesture data as well as facial expressions.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Vogt, P.A. and Valdenegro Toro, M.A. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 17 Jun 2025 07:38 |
| Last Modified: | 17 Jun 2025 07:38 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/35373 |
Actions (login required)
 |
View Item |