Janicek, Jakub (2026) Retrieval-Augmented Generation over Czech E-Commerce: Comparing Keyword, Semantic, and Hybrid Retrieval. Bachelor's Thesis, Artificial Intelligence.
|
Text
BachelorThesisFinal.pdf Download (1MB) | Preview |
Abstract
Large commercial product catalogues pose distinct challenges for retrieval-augmented generation, with high inter-product semantic similarity and queries ranging from exact identifier lookups to broad need-based descriptions. Using a Czech automotive e-commerce catalogue of 177 thousand products, a synthetic evaluation set of 500 query-answer pairs per category was constructed across three categories: SKU-explicit, name-based, and need-based. BM25, dense, and hybrid retrieval were evaluated with and without cross-encoder reranking using Precision at k, Recall at k, Hit at k, and MRR at k. BM25 and hybrid retrieval achieved near-perfect Hit at 20 of 0.988 and 0.994 on SKU-explicit queries, while performance dropped substantially for name-based and need-based queries with MRR at 20 below 0.55. Reranking consistently improved MRR across all methods and categories. End-to-end RAG evaluation, using an LLM-as-a-judge, identified retrieval failure as the dominant failure mode, suggesting that evidence retrieval rather than generation quality is the primary bottleneck. These findings indicate that off-the-shelf retrieval configurations may be insufficient for semantic retrieval over large, identifier-heavy, and high- similarity product catalogues, motivating further research on domain-specific embedding models and product-aware retrieval architectures.
| Item Type: | Thesis (Bachelor's Thesis) |
|---|---|
| Supervisor name: | Tashu, T.M. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Bachelor's Thesis |
| Language: | English |
| Date Deposited: | 16 Jul 2026 12:31 |
| Last Modified: | 16 Jul 2026 12:31 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/37434 |
Actions (login required)
 |
View Item |