Cortild, Daniel (2025) New Tight Bounds for SGD without Variance Assumption: A Computer-Aided Lyapunov Analysis. Master's Thesis / Essay, Applied Mathematics.
|
Text
mAppM2025CortildD.pdf Download (1MB) | Preview |
|
![[img]](https://fse.studenttheses.ub.rug.nl/style/images/fileicons/text.png) |
Text
Toestemming.pdf Restricted to Registered users only Download (193kB) |
Abstract
The analysis of Stochastic Gradient Descent (SGD) often relies on making some assumption on the variance of the stochastic gradients, which is usually not satisfied or difficult to verify in practice. This work contributes to a recent line of works which attempt to provide guarantees without making any variance assumption, leveraging only the (strong) convexity and smoothness of the loss functions. In this context, we prove new theoretical bounds derived from the monotonicity of a simple Lyapunov energy, improving the current state-of-the-art and extending their validity to larger step-sizes. Our theoretical analysis is backed by a Performance Estimation Problem analysis, which allows us to claim that, empirically, the bias term in our bounds is tight within our framework.
| Item Type: | Thesis (Master's Thesis / Essay) |
|---|---|
| Supervisor name: | Peypouquet, J.G. and Camlibel, M.K. |
| Degree programme: | Applied Mathematics |
| Thesis type: | Master's Thesis / Essay |
| Language: | English |
| Date Deposited: | 11 Jun 2025 06:49 |
| Last Modified: | 11 Jun 2025 06:49 |
| URI: | https://fse.studenttheses.ub.rug.nl/id/eprint/35343 |
Actions (login required)
 |
View Item |