From 2ae15f3cc6e4003b7ed219c3146a49221a81764e Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sun, 15 Nov 2026 06:04:21 +0000 Subject: [PATCH] Fix DDPM covariance notation and noise-scale prose in unit 1 notebooks Match q(x_t | x_0) in notebook 02 to notebook 01 by using covariance (1 - bar-alpha_t) I instead of a square-root on the covariance, and describe the forward step as adding noise scaled by sqrt(beta_t). Co-authored-by: Charles --- unit1/01_introduction_to_diffusers.ipynb | 2 +- unit1/02_diffusion_models_from_scratch.ipynb | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/unit1/01_introduction_to_diffusers.ipynb b/unit1/01_introduction_to_diffusers.ipynb index ba1bf2b..5c3aa19 100644 --- a/unit1/01_introduction_to_diffusers.ipynb +++ b/unit1/01_introduction_to_diffusers.ipynb @@ -692,7 +692,7 @@ "q(\\mathbf{x}_{1:T} \\vert \\mathbf{x}_0) = \\prod^T_{t=1} q(\\mathbf{x}_t \\vert \\mathbf{x}_{t-1})$

\n", "\n", "\n", - "That is, we take $x_{t-1}$, scale it by $\\sqrt{1 - \\beta_t}$ and add noise scaled by $\\beta_t$. This $\\beta$ is defined for every t according to some schedule, and determines how much noise is added per timestep. Now, we don't necessarily want to do this operation 500 times to get $x_{500}$ so we have another formula to get $x_t$ for any t given $x_0$:

\n", + "That is, we take $x_{t-1}$, scale it by $\\sqrt{1 - \\beta_t}$ and add noise scaled by $\\sqrt{\\beta_t}$. This $\\beta$ is defined for every t according to some schedule, and determines how much noise is added per timestep. Now, we don't necessarily want to do this operation 500 times to get $x_{500}$ so we have another formula to get $x_t$ for any t given $x_0$:

\n", "\n", "$\\begin{aligned}\n", "q(\\mathbf{x}_t \\vert \\mathbf{x}_0) &= \\mathcal{N}(\\mathbf{x}_t; \\sqrt{\\bar{\\alpha}_t} \\mathbf{x}_0, {(1 - \\bar{\\alpha}_t)} \\mathbf{I})\n", diff --git a/unit1/02_diffusion_models_from_scratch.ipynb b/unit1/02_diffusion_models_from_scratch.ipynb index 70b3c94..38f687b 100644 --- a/unit1/02_diffusion_models_from_scratch.ipynb +++ b/unit1/02_diffusion_models_from_scratch.ipynb @@ -1484,10 +1484,10 @@ "q(\\mathbf{x}_{1:T} \\vert \\mathbf{x}_0) = \\prod^T_{t=1} q(\\mathbf{x}_t \\vert \\mathbf{x}_{t-1})$

\n", "\n", "\n", - "That is, we take $x_{t-1}$, scale it by $\\sqrt{1 - \\beta_t}$ and add noise scaled by $\\beta_t$. This $\\beta$ is defined for every t according to some schedule, and determines how much noise is added per timestep. Now, we don't necessarily want to do this operation 500 times to get $x_{500}$ so we have another formula to get $x_t$ for any t given $x_0$:

\n", + "That is, we take $x_{t-1}$, scale it by $\\sqrt{1 - \\beta_t}$ and add noise scaled by $\\sqrt{\\beta_t}$. This $\\beta$ is defined for every t according to some schedule, and determines how much noise is added per timestep. Now, we don't necessarily want to do this operation 500 times to get $x_{500}$ so we have another formula to get $x_t$ for any t given $x_0$:

\n", "\n", "$\\begin{aligned}\n", - "q(\\mathbf{x}_t \\vert \\mathbf{x}_0) &= \\mathcal{N}(\\mathbf{x}_t; \\sqrt{\\bar{\\alpha}_t} \\mathbf{x}_0, \\sqrt{(1 - \\bar{\\alpha}_t)} \\mathbf{I})\n", + "q(\\mathbf{x}_t \\vert \\mathbf{x}_0) &= \\mathcal{N}(\\mathbf{x}_t; \\sqrt{\\bar{\\alpha}_t} \\mathbf{x}_0, {(1 - \\bar{\\alpha}_t)} \\mathbf{I})\n", "\\end{aligned}$ where $\\bar{\\alpha}_t = \\prod_{i=1}^T \\alpha_i$ and $\\alpha_i = 1-\\beta_i$

\n", "\n", "The maths notation always looks scary! Luckily the scheduler handles all that for us (uncomment the next cell to check out the code). We can plot $\\sqrt{\\bar{\\alpha}_t}$ (labelled as `sqrt_alpha_prod`) and $\\sqrt{(1 - \\bar{\\alpha}_t)}$ (labelled as `sqrt_one_minus_alpha_prod`) to view how the input (x) and the noise are scaled and mixed across different timesteps:\n"