Diffusion model: Difference between revisions

Revision as of 03:27, 16 April 2025

Revision as of 16:33, 16 May 2025

Line 1:

{{Short description|Deep learning algorithm}}{{About|the technique in generative statistical modeling|3=Diffusion (disambiguation)}}{{Machine learning|Artificial neural network}}

In [[machine learning]], '''diffusion models''', also known as '''diffusion-based probabilisticgenerative models''' or '''score-based generative models''', are a class of [[latent variable model|latent variable]] [[generative model|generative]] models. A diffusion model consists of threetwo major components: the forward process, the reversediffusion process, and the reverse sampling procedure.<ref name="chang23design">{{cite arXiv |last1=Chang |first1=Ziyi |last2=Koulieris |first2=George Alex |last3=Shum |first3=Hubert P. H. |title=On the Design Fundamentals of Diffusion Models: A Survey |date=2023 |eprint=2306.04542 |class=csprocess.LG}}</ref> The goal of diffusion models is to learn a [[diffusion process]] for a given dataset, such that the process can generate new elements that are distributed similarly as the original dataset. A diffusion model models data as generated by a diffusion process, whereby a new datum performs a [[Wiener process|random walk with drift]] through the space of all possible data.<ref name="song"/> A trained diffusion model can be sampled in many ways, with different efficiency and quality.

There are various equivalent formalisms, including [[Markov chain]]s, denoising diffusion probabilistic models, noise conditioned score networks, and stochastic differential equations.<ref>{{cite journal |last1=Croitoru |first1=Florinel-Alin |last2=Hondru |first2=Vlad |last3=Ionescu |first3=Radu Tudor |last4=Shah |first4=Mubarak |date=2023 |title=Diffusion Models in Vision: A Survey |journal=IEEE Transactions on Pattern Analysis and Machine Intelligence |volume=45 |issue=9 |pages=10850–10869 |arxiv=2209.04747 |doi=10.1109/TPAMI.2023.3261988 |pmid=37030794 |s2cid=252199918}}</ref> They are typically trained using [[Variational Bayesian methods|variational inference]].<ref name="ho" /> The model responsible for denoising is typically called its "[[#Choice of architecture|backbone]]". The backbone may be of any kind, but they are typically [[U-Net|U-nets]] or [[Transformer (deep learning architecture)|transformers]].