Borszukovszki, Mirko (2024) Know what you don't know: Uncertainty Estimation on Corrupted Images in Visual Language Models. Bachelor's Thesis, Artificial Intelligence.
|
Text
BachelorThesismborszukovszki.pdf Download (2MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
Toestemming Borszukovski.pdf Restricted to Staff only Download (131kB) |
Abstract
To leverage the full potential of Large Language Models (LLMs) in various tasks like machine translation, programming, summarising large documents, automating customer service or even general-purpose chatbot assistants, it is crucial to have some information on their answers’ uncertainty. This means that the model has to be able to quantify how certain it is in the correctness of a given response. Bad uncertainty estimates can lead to confident wrong answers undermining trust in these models thus preventing practical applications. Quite a lot of research has been done on language models that work with text inputs and provide text outputs. Still, since the visual capabilities have been added to these models recently, there has not been much progress on the uncertainty of Visual Language Models (VLMs). This thesis aims to further our understanding of this topic. We tested three state-of-the-art VLMs on corrupted image data. We found that the severity of the corruption negatively impacted the models’ ability to estimate their uncertainty and the models also showed overconfidence in most of the experiments.
| Item Type: | Thesis (Bachelor's Thesis) |
|---|---|
| Supervisor name: | Valdenegro Toro, M.A. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Bachelor's Thesis |
| Language: | English |
| Date Deposited: | 06 Aug 2024 14:01 |
| Last Modified: | 08 Aug 2024 14:13 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/33737 |
Actions (login required)
 |
View Item |