Conjugate Inference: Multivariate Gaussian Likelihood

Domain: \[\vec{x}\in\mathbb{R}^d\]

Parameters: The Gaussian likelihood is parametrised by its mean vector, \(\vec{\mu}\), and precision matrix, \(\Lambda\).
\[\Theta = \{\vec{\mu}, \Lambda\}\]

Likelihood: \[P(\vec{x}|\Theta) = N(\vec{x}|\vec{\mu},\Lambda^{-1})\]

Prior: A normal–Wishart distribution.
\[P(\Theta) = N(\vec{\mu}|\vec{\eta}_0,(\tau_0\Lambda)^{-1})\ W(\Lambda|V_0,\nu_0)\]
The probability density function of the Wishart distribution is
\[W(\Lambda|V,\nu) = \frac{|\Lambda|^{(\nu-d-1)/2}\exp\left(-\frac{1}{2}\textrm{Trace}(V^{-1}\Lambda)\right)}{2^{\nu d/2}|V|^{\nu/2}\Gamma_d(\nu/2)}\]
where
\[\Gamma_d(\nu/2) = \pi^{d(d-1)/4}\ \prod_{i=1}^d \Gamma\left(\frac{\nu-i+1}{2}\right)\]
is the multivariate gamma function.

Posterior:
\[P(\Theta|D) = N(\vec{\mu}|\vec{\eta}_1,(\tau_1\Lambda)^{-1})\ W(\Lambda|V_1,\nu_1)\]
with
\begin{eqnarray}
\vec{\eta}_1 &=& \frac{\tau_0\vec{\eta}_0 + \vec{S}^{(1)}}{\tau_1} \\
\tau_1 &=& \tau_0 + S^{(0)} \\
\nu_1 &=& \nu_0 + S^{(0)} \\
V_1^{-1} &=& V_0^{-1} + S^{(2)} + \tau_0\vec{\eta}_0^{I\!I} - \tau_1\vec{\eta}_1^{I\!I}
\end{eqnarray}
where
\begin{eqnarray}
S^{(0)} &=& |D| \\
\vec{S}^{(1)} &=& \sum_{\vec{x}\in D} \vec{x} \\
S^{(2)} &=& \sum_{\vec{x}\in D} \vec{x}^{I\!I}
\end{eqnarray}
and \(\vec{x}^{I\!I} \equiv \vec{x}\vec{x}^T\) is the outer product of a vector with itself.

Marginal likelihood:
\[P(D) =
\pi^{-S^{(0)}d/2}
\left(\frac{\tau_0}{\tau_1}\right)^{d/2}
\frac{|V_1|^{\nu_1/2}}{|V_0|^{\nu_0/2}}
\ \prod_{i=1}^d \frac{\Gamma\left((\nu_1-i+1)/2\right)}{\Gamma\left((\nu_0-i+1)/2\right)}
\]