Conjugate Inference: Gaussian Likelihood

Domain: Here we consider only univariate normal distributions. \[x\in\mathbb{R}\]

Parameters: The Gaussian likelihood is parametrised by its mean, \(\mu\), and inverse variance, \(\lambda\).
\[\Theta = \{\mu, \lambda\}\]

Likelihood: \[P(x|\Theta) = N(x|\mu,\lambda^{-1})\]

Prior: A normal–gamma distribution.
\[P(\Theta) = N(\mu|\eta_0,(\tau_0\lambda)^{-1})\ G(\lambda|\alpha_0,\beta_0)\]
Note that the following parametrisation of the gamma distribution is used:
\[G(x|\alpha,\beta) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}\]
(There is another parametrisation, which uses \(\beta^{-1}\) rather than \(\beta\)).

Posterior:
\[P(\Theta|D) = N(\mu|\eta_1,(\tau_1\lambda)^{-1})\ G(\lambda|\alpha_1,\beta_1)\]
with
\begin{eqnarray}
\eta_1 &=& \frac{\eta_0\tau_0 + S^{(1)}}{\tau_1} \\
\tau_1 &=& \tau_0 + S^{(0)} \\
\alpha_1 &=& \alpha_0 + \frac{S^{(0)}}{2} \\
\beta_1 &=& \beta_0 + \frac{1}{2}\left( S^{(2)}+\eta_0^2\tau_0 - \eta_1^2\tau_1 \right)
\end{eqnarray}
where
\begin{eqnarray}
S^{(0)} &=& |D| \\
S^{(1)} &=& \sum_{x\in D} x \\
S^{(2)} &=& \sum_{x\in D} x^2
\end{eqnarray}

Marginal likelihood:
\[P(D) =
(2\pi)^{-S^{(0)}/2}
\frac{\sqrt{\tau_0} \beta_0^{\alpha_0} \Gamma(\alpha_1)}
{\sqrt{\tau_1} \beta_1^{\alpha_1} \Gamma(\alpha_0)} \]