Optimizing LLMs for Persuasion Improves Generalization

Reedi, Aksel Joonas (2025) Optimizing LLMs for Persuasion Improves Generalization. Bachelor's Thesis, Artificial Intelligence.

Preview	Text bAI2025AkselJoonasReedi.pdf Download (932kB) | Preview
	Text toestemming reedi.pdf Restricted to Registered users only Download (173kB)

Abstract

Large Language Models (LLMs) are typically optimized for truthfulness, yet recent work shows this approach is prone to overfit, yielding brittle reasoning that struggles to generalize to unseen contexts. We introduce persuasion-based training as an alternative to truth-based optimization, demonstrating its potential for improving model generalization through evolutionary prompt optimization. Our experimental setup involves two LLMs debating on a question, while a third LLM acts as a judge to select the debate winner. We use a quality-diversity (QD) framework to optimize these debate prompts across seven persuasion families (rationality, authority, emotional appeal, etc.) over several debate tournaments. Across three model scales (7B, 32B, 72B parameters) and multiple dataset sizes, persuasion-optimized strategies consistently outperform truth-optimized ones, showing greater ability to generalize to unseen questions. Persuasion also matches or surpasses truth optimization performance on test set questions, suggesting superior transfer to new contexts. These results indicate that competitive pressure to convince, rather than collaborate toward correctness, may foster more transferable reasoning skills. Our framework offers a method for comparing alignment objectives and highlights persuasiveness as a promising lever for improving LLM generalization.

Item Type:	Thesis (Bachelor's Thesis)
Supervisor name:	Pourcel, G. A.
Degree programme:	Artificial Intelligence
Thesis type:	Bachelor's Thesis
Language:	English
Date Deposited:	02 Sep 2025 12:26
Last Modified:	02 Sep 2025 12:26
URI:	https://fse.studenttheses.ub.rug.nl/id/eprint/36937

Actions (login required)

View Item