Radu, Stefania (2024) Uncertainty in Semantic Language Modeling with PIXELS. Master's Thesis / Essay, Artificial Intelligence.
|
Text
mAI2024RaduMS.pdf Download (13MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
toestemming_ Stefania Radu _ degree programme_ Artificial Intelligence.pdf Restricted to Registered users only Download (135kB) |
Abstract
Traditional Language Models like BERT are trained using raw text, split into separate chunks using a tokenizer. These models suffer from 3 main challenges: a lack of context understanding, a bottleneck in the vocabulary, and unreliable predictions caused by high epistemic uncertainty. This study investigates a new approach – Visual Language Models (VLMs) – by rendering text as an image and replacing the masked language modeling task with patch reconstruction at the pixel level. The novelty of this work consists of analysing uncertainty and confidence in VLMs models across 18 languages and 7 scripts, all part of 3 semantically challenging tasks: Named Entity Recognition (NER), Sequence Classification (SC), and Question-Answering (QA). This is achieved through several Uncertainty Quantification methods such as Monte Carlo Dropout, Transformer Attention, and Ensemble Learning. The results suggest that VLMs underestimate uncertainty when reconstructing patches, especially when a large proportion of the image is masked. The uncertainty is also influenced by the script, with Latin languages displaying lower uncertainty, compared to the Geez or Chinese Characters scripts. The findings on ensemble learning show better performance when applying hyperparameter tuning during the NER and QA tasks across 16 languages, as well as improved overall calibration.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Valdenegro Toro, M.A. and Zullich, M. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 17 Jul 2024 14:06 |
| Last Modified: | 17 Jul 2024 14:06 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/33497 |
Actions (login required)
 |
View Item |