Ivanova, Iva (2026) A Cross-Task Evaluation of Single-Pass Generation versus Iterative Self-Refinement in LLM Workflows. Bachelor's Thesis, Artificial Intelligence.
|
Text
ivanovafinalthesis.pdf Download (1MB) | Preview |
Abstract
This thesis evaluates whether iterative self-refinement improves measured LLM performance compared to single-pass generation across summarization, constrained summarization, and code optimization. The gemma3:4b model generated an initial output, provided feedback on its own response, and revised it for 50 rounds without external verification. Outputs were evaluated at the single-pass baseline and at refinement rounds 1, 3, 5, 10, and 50. The study used 1000 summarization examples, 100 constrained summarization examples, and 28 code optimization examples. Summarization was evaluated using cosine similarity, ROUGE-Lsum F1, and BERTScore F1. Code optimization was evaluated using cosine similarity to reference solutions, while constrained summarization used an LLM-as-a-judge framework and a fixed word-count rule. All summarization metrics decreased significantly across refinement rounds. Code optimization cosine similarity also decreased, but the differences were not statistically significant after Holm adjustment. For constrained summarization, the average completion ratio remained high and stable. Refinement improved tone and style, mostly after only one round, but reduced length compliance and the overall pass rate. Under the specific model, prompts, settings, tasks, and evaluation methods used in this study, iterative self-refinement did not outperform single-pass generation overall. Its effects were task-specific, constraint-specific, and metric-dependent.
| Item Type: | Thesis (Bachelor's Thesis) |
|---|---|
| Supervisor name: | Tashu, T.M. |
| Degree programme: | Artificial Intelligence |
| Thesis type: | Bachelor's Thesis |
| Language: | English |
| Date Deposited: | 16 Jul 2026 12:35 |
| Last Modified: | 16 Jul 2026 12:35 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/37435 |
Actions (login required)
 |
View Item |