Al Hamzi, Nadir (2022) Parameter efficient transfer learning and fine-tuning of BERT-based transformers for Arabic QA. Master's Thesis / Essay, Artificial Intelligence.
|
Text
mAI_2022_NadirAlHamzi.pdf Download (1MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
toestemming.pdf Restricted to Registered users only Download (147kB) |
Abstract
Most advances in NLP and efforts in building large, annotated datasets have been focused on English. Other languages, such as Arabic, lack equivalent resources despite being one of the most spoken languages in the world. This study aims to establish different benchmarks for open-domain Arabic QA. It investigates a range of transfer learning strategies to address the challenges of training an IR-based QA system with three modules; retrieval, re-ranking, and reading comprehension (RC). ElasticSearch is employed to index the collections for fast retrieval configured with the BM25 baseline for the initial stage. Parameter efficient transfer learning and fine-tuning transfer learning approaches are explored for the passage re-ranking and RC tasks. Different monolingual and cross-lingual models, including mBERT, AraBERT, ARBERT, MARBER, and XLM-Roberta, are examined under two settings: adapter tuning and fine-tuning. The results reveal that monolingual models, in general, are superior to cross-lingual models. ARBERT achieves the highest score in passage re-ranking tasks and can serve as a baseline for re-ranking, while AraBERT offers a better benchmark for RC. Furthermore, adapter tuning substantially reduces the model’s size and speeds up training procedures without significantly impacting performance. It proves to be a viable alternative to traditional fine-tuning techniques.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Mohades Kasaei, S.H. and Spenader, J.K. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 01 Apr 2022 09:11 |
| Last Modified: | 01 Apr 2022 09:11 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/26666 |
Actions (login required)
 |
View Item |