Any-Order Flexible Length Masked Diffusion Aug 2025
With Jaeyeon Kim, Carles Domingo-Enrich, Sham Kakade, Yilun Du, Timothy Ngotiaoco, Sitan Chen, and Michael Albergo
We introduce FlexMDM, a class of masked diffusion models for variable-length data based on joint interpolant, an extension of the stochastic interpolant framework.
We further show that FlexMDM retains the any-order sampling guarantees of masked diffusion as established in prior work by Jaeyeon Kim and Kulin Shah.
With wonderful co-author Jaeyeon Kim, we demonstrate that the method scales to 8B parameters and achieves notable performance improvements over previous masked diffusion models.
Also see concurrent work EditFlow.
Debiasing Guidance with Sequential Monte Carlo Jan 2025
With Paul Jeha, Jes Frellsen, Michael Albergo, Pietro Lio, and Francisco Vargas
Guidance in diffusion models are biased. By using a specific choice of intermediate distribution, this bias can be corrected with Sequential Monte Carlo without extra function evaluations.
Also see concurrent work Feynman-Kac Correctors for a similar scheme, and follow-up work Radon-Nikodym Estimators for a more systematic approach to derive Sequential Monte Carlo schemes for general targets.