prior knowledge

• Prob & Stats

  • $X = N(\mu, \Sigma)$ mean, variance 로 이루어진 distribution
    • $\varepsilon \sim N(0, I)$ gaussian distribution
  • $E(aX + B) = aE(x)+b$
  • $V(aX + b) = a^2V(X)$
  • $\sigma(aX + b) = |a|\sigma (X)$ standard deviation^2 = var
  • $P(B|A) = P(A |B){P(B) \over P(A)}$

  Bayes' theorem

• Distribution

  • $q(x): real \ distribution$
  • $p_{\theta}(x) : network \ distribution$

• Advanced

  • $N(x: \mu, {\sigma}^2) = {1 \over \sigma \sqrt {2\pi}}e^{- {{(x-\mu)^2} \over {2 {\sigma^2}}}} = f(x)$ probability density function (pdf)

  $\sigma^2$ = standard deviation = $var^2$

  • $T_f(x) = \sum^{\infty}_{n=0}{{f^{(n)}(a)}\over {n!}}(x-a)^n = f(a) + f'(a)(x-a) + {1 \over 2}f''(a)(x -a)^2 + ...$
    • $\approx f(a) + f '(a)(x -a)$

  Taylor series

• prob

$P(D|A,B,C)={{P(A|B,C,D)P(B,C,D)}\over{P(A,B,C)}}$

$P(D|A,B)={{P(A|B,D)P(B,D)}\over{P(A,B)}}$

$= P(A|B,D){{P(B,D)}\over{P(A,B)}}$

$=P(A|B,D){{P(D|B)P(B)}\over{P(A|B)P(B)}}$

$=P(A|B,D){{P(D|B)}\over{P(A|B)}}$

forward process

forward pass

forward trajectory

intractability

다루기 힘듦

Tractability

다루기 쉬움

What are Diffusion Models?