Lee Cheuk-KitBrian

@ Publications

Debiasing Guidance with Sequential Monte Carlo Jan 2025

With Paul JehaJes FrellsenMichael AlbergoPietro Lio, and Francisco Vargas

Discrete diffusion models are a class of generative models that produce samples from an approximated data distribution within a discrete state space. Often, there is a need to target specific regions of the data distribution. Current guidance methods aim to sample from a distribution with mass proportional to p0(x0)p(ζx0)αp_0(x_0)p(\zeta|x_0)^\alpha but fail to achieve this in practice. We introduce a Sequential Monte Carlo algorithm that generates unbiasedly from this target distribution, utilising the learnt unconditional and guided process. We validate our approach on low-dimensional distributions, controlled images and text generations. For text generation, our method provides strong control while maintaining low perplexity compared to guidance-based approaches.

Debiasing Guidance with Sequential Monte Carlo preview image