Gentile, Daniel (2025) Human-Centered Epistemic Uncertainties for Handwritten Digits. Bachelor's Thesis, Artificial Intelligence.
|
Text
bAI2025GentileD.pdf Download (3MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
Toestemming.pdf Restricted to Registered users only Download (179kB) |
Abstract
Uncertainty Quantification (UQ) helps build trustworthy and interpretable neural networks. Most prior work uses synthetic or out-of-distribution data, which fails to reflect real-world challenges. In an attempt to bridge this gap, this thesis proposes a novel human-generated dataset of handwritten digits to analyze the effects of different UQ methods on user perception of model confidence and trust. A web-based platform was built for users to view model predictions, assess confidence scores, and give Likert-scale feedback. We compared three different models: a simple convolutional neural network (CNN) with softmax for confidence estimation, a Monte Carlo Dropout model, and a Deep Ensemble model. Each model was compared in terms of accuracy, confidence calibration, and their effects on user trust. The results show that even though the three models were equally accurate, the baseline model had a much higher level of confidence—usually bordering on overconfidence—when compared with the UQ models. Bayesian analysis also supported the fact that the baseline model was poorly calibrated, with strong evidence of being overconfident. User feedback on Likert-scale prompts did not show statistically significant differences, though the patterns indicated that the users perceived the confidence of the baseline model as being more extreme. This work provides a dataset, process, and findings that support future studies of human-centered AI in real-world tasks.
| Item Type: | Thesis (Bachelor's Thesis) |
|---|---|
| Supervisor name: | Jong, I.P. de and Valdenegro Toro, M.A. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Bachelor's Thesis |
| Language: | English |
| Date Deposited: | 06 May 2025 07:52 |
| Last Modified: | 06 May 2025 07:52 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/35138 |
Actions (login required)
 |
View Item |