PhD Position F/M Theory for Discrete Diffusion Models

id="offer-description" class="ecl-u-type-heading-2 ecl-u-mt-none ecl-u-mb-l">Offer Description The PhD position will be in the framework of the ERC Starting Grant DYNASTY (Dynamics-Aware Theory of Deep Learning). The position might include traveling to conferences for paper presentation. Travel expenses will be covered within the limits of the scale in force. Discrete diffusion models have become a leading alternative to autoregressive models for generating text and other categorical data [1, 2]. They generate samples by reversing a Markov chain that progressively corrupts the data, using a score function learned by a neural network [1]. Despite their empirical success, their theoretical understanding remains far less developed than that of their continuous counterparts. The objective of this PhD is to study the theoretical foundations of discrete diffusion models, and in particular how well they learn and generalize when the score is estimated by a neural network. Building on recent statistical analyses [3, 4] and on generalization bounds for diffusion models [5], the project will investigate how statistical, approximation and optimization errors affect the quality of the generated samples, and will validate the theory through numerical experiments. References. [1] A. Lou, C. Meng, S. Ermon. Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution. ICML, 2024. [2] S. S. Sahoo, M. Arriola, Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, V. Kule