diff options
Diffstat (limited to 'finale/sections')
| -rw-r--r-- | finale/sections/bayesian.tex | 24 |
1 files changed, 14 insertions, 10 deletions
diff --git a/finale/sections/bayesian.tex b/finale/sections/bayesian.tex index efc1526..1426c97 100644 --- a/finale/sections/bayesian.tex +++ b/finale/sections/bayesian.tex @@ -104,17 +104,21 @@ where $\mathcal{N}^+(\cdot)$ is a gaussian truncated to lied on $\mathbb{R}^+$ since $\Theta$ is a transformed parameter $z \mapsto -\log(1 - z)$. This model is represented in the graphical model of Figure~\ref{fig:graphical}. -The product-form of the prior implies that the KL term is entirely decomposable: +The product-form of the prior implies that the KL term is entirely decomposable. +Since an easy closed-form formula exists for the KL divergence between two +gaussians, we approximate the truncated gaussians by their non-truncated +counterpart. \begin{equation} - \text{KL}(q_{\mathbf{\Theta'}}, p_{\mathbf{\Theta}}) = \sum_{ij} + \label{eq:kl} + \begin{split} + \text{KL}(q_{\mathbf{\Theta'}}, p_{\mathbf{\Theta}}) &= \sum_{ij} KL\left(\mathcal{N}^+(\mu_{ij}, \sigma_{ij}), \mathcal{N}^+(\mu^0_{ij}, - \sigma^0_{ij})\right) + \sigma^0_{ij})\right) \\ + &\approx \sum_{ij} \log \frac{\sigma^0_{ij}}{\sigma_{ij}} + + \frac{\sigma^2_{ij} + {(\mu_{ij} - \mu_{ij}^0)}^2}{2{(\sigma^0_{ij})}^2} + \end{split} \end{equation} -ince an easy closed-form formula exists for the KL divergence between two -gaussians, we approximate the truncated gaussians by their non-truncated -counterpart. \begin{equation} - \label{eq:kl} - \text{KL}(q_{\mathbf{\Theta'}}, p_{\mathbf{\Theta}}) \approx \sum_{i,j} \log - \frac{\sigma^0_{ij}}{\sigma_{ij}} + -\end{equation} +Reparametrization trick +Batches +Algorithm |