Gaussian Bivariate Distribution

The Gaussian bivariate distribution is given by

$$P(x_1,x_2) = {1\over 2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}\, \mathop{\rm exp}\nolimits \left[{-{z\over 2(1-\rho^2)}}\right],$$ (1)

where

$$z\equiv {(x_1-\mu_1)^2\over{\sigma_1}^2} - {2\rho(x_1-\mu_1)... ...\mu_2)\over\sigma_1\sigma_2}+{(x_2-\mu_2)^2\over{\sigma_2}^2},$$ (2)

and

$$\rho \equiv \mathop{\rm cov}\nolimits (x_1,x_2) = {\left\lan... ...rangle{}\left\langle{x_2}\right\rangle{}\over\sigma_1\sigma_2}$$ (3)

is the Covariance. Let $X_1$ and $X_2$ be normally and independently distributed variates with Mean 0 and Variance 1. Then define

$$Y_1 \equiv \mu_1+\sigma_{11}X_1+\sigma_{12}X_2$$ (4)

$$Y_2 \equiv \mu_2+\sigma_{21}X_1+\sigma_{22}X_2.$$ (5)

These new variates are normally distributed with Mean $\mu_1$ and $\mu_2$, Variance

$${\sigma_1}^2 \equiv {\sigma_{11}}^2+{\sigma_{12}}^2$$ (6)

$${\sigma_2}^2 \equiv {\sigma_{21}}^2+{\sigma_{22}}^2,$$ (7)

and Covariance

$$V_{12}\equiv \sigma_{11}\sigma_{21}+\sigma_{12}\sigma_{22}.$$ (8)

The Covariance matrix is

$$V_{ij} = \left[{\matrix{{\sigma_1}^2 & \rho\sigma_1\sigma_2\cr \rho\sigma_1\sigma_2 & {\sigma_2}^2\cr}}\right],$$ (9)

where

$$\rho\equiv {{V_{12}}\over {\sigma_1}{\sigma_2}} ={\sigma_{11}\sigma_{21}+\sigma_{12}\sigma_{22}\over {\sigma_1}{\sigma_2}}.$$ (10)

The joint probability density function for $x_1$ and $x_2$ is

$$f(x_1,x_2)\,dx_1\,dx_2 = {1\over 2\pi} e^{-({x_1}^2+{x_2}^2)/2} \,dx_1\,dx_2.$$ (11)

However, from (4) and (5) we have

$$\left[{\matrix{y_1-\mu_1\cr y_2-\mu_2\cr}}\right] = \left[{\... ...\sigma_{22}\cr}}\right] \left[{\matrix{x_1\cr x_2\cr}}\right].$$ (12)

Now, if

$$\left\vert\matrix{\sigma_{11} & \sigma_{12}\cr \sigma_{21} & \sigma_{22}\cr}\right\vert \not=0,$$ (13)

then this can be inverted to give

$$\left[\begin{array}{c}x_1\\ x_2\end{array}\right] = \left[\begin{array}{cc}\sigma_{11} & \sigma_{12}\\ \sigma_{21} &... ...t]^{-1} \left[\begin{array}{c}y_1-\mu_1\\ y_2-\mu_2\end{array}\right]\nonumber$$

$$= {1\over \sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21}} \left[\beg... ...right] \left[\begin{array}{c}y_1-\mu_1\\ y_2-\mu_2\end{array}\right].\nonumber$$ (14)

Therefore,

$${x_1}^2+{x_2}^2 = {[\sigma_{22}(y_1-\mu_1)-\sigma_{12}(y_2-\... ..._2)]^2\over(\sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21})^2}.$$ (15)

Expanding the Numerator gives

${\sigma_{22}}^2(y_1-\mu_1)^2-2\sigma_{12}\sigma_{22}(y_1-\mu_1)(y_2-\mu_2)$
$+{\sigma_{12}}^2(y_2-\mu_2)^2+{\sigma_{21}}^2(y_1-\mu_1)^2$
$-2\sigma_{11}\sigma_{21}(y_1-\mu_1)(y_2-\mu_2)+{\sigma_{11}}^2(y_2-\mu_2)^2,$
	(16)

so

$({x_1}^2+{x_2}^2)(\sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21})^2$
$= (y_1-\mu_1)^2({\sigma_{21}}^2+{\sigma_{22}}^2) -2(y_1-\mu_1)(y_2-\mu_2)(\sigma_{11}\sigma_{21}+\sigma_{12}\sigma_{22})$
$\quad + (y_2-\mu_2)^2({\sigma_{11}}^2+{\sigma_{12}}^2)$
$= {\sigma_2}^2(y_1-\mu_1)^2-2(y_1-\mu_1)(y_2-\mu_2)(\rho\sigma_1\sigma_2)+{\sigma_1}^2(y_2-\mu_2)^2$
$= {\sigma_1}^2{\sigma_2}^2\left[{{(y_1-\mu_1)^2\over {\sigma_1}^2} -{2\rho(y_1-... ...1)(y_2-\mu_2)\over \sigma_1\sigma_2}+{(y_2-\mu_2)^2\over {\sigma_2}^2}}\right].$	(17)

But

$${1\over 1-\rho^2} = {1\over 1-{{V_{12}}^2\over {\sigma_1}^2{... ...a_{22}}^2)-(\sigma_{11}\sigma_{21}+\sigma_{12}\sigma_{22})^2}.$$ (18)

The Denominator is

${\sigma_{11}}^2{\sigma_{21}}^2+{\sigma_{11}}^2{\sigma_{22}}^2+{\sigma_{12}}^2{\sigma_{21}}^2+{\sigma_{12}}^2{\sigma_{22}}^2-{\sigma_{11}}^2{\sigma_{21}}^2$
$-2\sigma_{11}\sigma_{12}\sigma_{21}\sigma_{22}-{\sigma_{12}}^2{\sigma_{22}}^2 = (\sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21})^2,\quad$	(19)

so

$${1\over 1-\rho^2} = {{\sigma_1}^2{\sigma_2}^2\over (\sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21})^2}$$ (20)

and

$${x_1}^2+{x_2}^2={1\over 1-\rho^2}\left[{{(y_1-\mu_1)^2\over ... ...r \sigma_1\sigma_2}+{(y_2-\mu_2)^2\over {\sigma_2}^2}}\right].$$ (21)

Solving for $x_1$ and $x_2$ and defining

$$\rho'\equiv {\sigma_1\sigma_2\sqrt{1-\rho^2}\over \sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21}}$$ (22)

gives

$$x_1 = {\sigma_{22}(y_1-\mu_1)-\sigma_{12}(y_2-\mu_2)\over\rho'}$$ (23)

$$x_2 = {-\sigma_{21}(y_1-\mu_1)+\sigma_{11}(y_2-\mu_2)\over\rho'}.$$ (24)

The Jacobian is

$$J\left({x_1,x_2\over y_1,y_2}\right) = \left\vert\begin{array}{cc}{\partial x_1\over\partial y_1} & {\pa... ...o'}\\ -{\sigma_{21}\over\rho'} & {\sigma_{11}\over\rho'}\end{array}\right\vert$$

$$= {1\over\rho'^2} (\sigma_{11}\sigma_{22}-\sigma_{12}\sigma_{21})$$

$$= {1\over\rho'} = {1\over\sigma_1\sigma_2\sqrt{1-\rho^2}}.$$ (25)

Therefore,

$$dx_1\,dx_2 = {dy_1\,dy_2\over \sigma_1\sigma_2\sqrt{1-\rho^2}}$$ (26)

and

$${1\over 2\pi} e^{-({x_1}^2+{x_2}^2)/2}\,dx_1\,dx_2 = {1\over 2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}} e^{-v/2}\,dy_1\,dy_2,$$ (27)

where

$$v\equiv {1\over 1-\rho^2}\left[{{(y_1-\mu_1)^2\over{\sigma_1... ...er\sigma_1\sigma_2}+{(y_2-\mu_2)^2\over {\sigma_2}^2}}\right].$$ (28)

Now, if

$$\left\vert\matrix{\sigma_{11} & \sigma_{12}\cr \sigma_{21} & \sigma_{22}\cr}\right\vert =0,$$ (29)

then

$$\sigma_{11}\sigma_{12}=\sigma_{12}\sigma_{21}$$ (30)

$$y_1 = \mu_1+\sigma_{11}x_1+\sigma_{12}x_2$$ (31)

$$y_2 = \mu_2+{\sigma_{12}\sigma_{21}\over \sigma_{11}} x_2 = \mu_2+{\sigma_{11}\sigma_{21}x_1 +\sigma_{12}\sigma_{21}x_2\over \sigma_{11}}$$

$$= \mu_2+{\sigma_{21}\over \sigma_{11}} (\sigma_{11}x_1+\sigma_{12}x_2),$$ (32)

so

$$y_1 = \mu_1+x_3$$ (33)

$$y_2 = \mu_2+{\sigma_{21}\over \sigma_{11}} x_3,$$ (34)

where

$$x_3=y_1-\mu_1 ={\sigma_{11}\over \sigma_{21}} (y_2-\mu_2).$$ (35)

The Characteristic Function is given by

$$\phi(t_1,t_2) \equiv \int_{-\infty}^\infty\int_{-\infty}^\infty e^{i(t_1x_1+t_2x_2)} P(x_1,x_2)\,dx_1\,dx_2$$

$$= N \int_{-\infty}^\infty \int_{-\infty}^\infty e^{i(t_1x_1+t_2x_2)} \mathop{\rm exp}\nolimits \left[{-{z\over 2(1-\rho^2)}}\right]\,dx_1\,dx_2,$$ (36)

where

$$z\equiv \left[{{(x_1-\mu_1)^2\over{\sigma_1}^2}} {-{2\rho(x_... ...ver\sigma_1\sigma_2}+{(x_2-\mu_2)^2\over {\sigma_2}^2}}\right]$$ (37)

and

$$N\equiv{1\over 2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}.$$ (38)

Now let

$$u \equiv x_1-\mu_1$$ (39)

$$w \equiv x_2-\mu_2.$$ (40)

Then

$$\phi(t_1,t_2) = N'\int_{-\infty}^\infty \left({e^{it_2w} \ma... ...^2}}\right]}\right)\int_{-\infty}^\infty e^v e^{t_1u}\,du\,dw,$$ (41)

where

$$v \equiv -{1\over 2(1-\rho^2)} {1\over {\sigma_1}^2} \left[{{u}^2-{2\rho\sigma_1w\over \sigma_2} u}\right]$$

$$N' \equiv {e^{i(t_1\mu_1+t_2\mu_2)}\over 2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}.$$ (42)

Complete the Square in the inner integral

$\int_{-\infty}^\infty \mathop{\rm exp}\nolimits \left\{{-{1\over 2(1-\rho^2)}{1... ...1}^2}\left[{{u}^2-{2\rho\sigma_1w\over\sigma_2} u}\right]}\right\} e^{t_1u}\,du$
$=\int_{-\infty}^\infty \mathop{\rm exp}\nolimits \left\{{-{1\over 2{\sigma_1}^2... ...ho^2)}\left({\rho_1\sigma_1w\over\sigma_2}\right)^2}\right\}e^{it_1u}\,du.\quad$	(43)

Rearranging to bring the exponential depending on $w$ outside the inner integral, letting

$$v\equiv u-\rho{\sigma_1w\over\sigma_2},$$ (44)

and writing

$$e^{it_1u}=\cos(t_1u)+i\sin(t_1u)$$ (45)

gives

$\phi(t_1,t_2)=N' \int_{-\infty}^\infty e^{it_2w}\mathop{\rm exp}\nolimits \left[{-{1\over 2{\sigma_2}^2(1-\rho^2)} w^2}\right]$
$\quad \times \mathop{\rm exp}\nolimits \left[{{\rho^2\over 2{\sigma_2}^2(1-\rho... ...ty\mathop{\rm exp}\nolimits \left[{-{1\over 2{\sigma_2}^2(1-\rho^2)}v^2}\right]$
$\quad \times \left\{{\cos\left[{t_1\left({v+{\rho\sigma_1w\over \sigma_2}}\righ... ...ft[{t_1\left({v+{\rho\sigma_1w\over \sigma_2}}\right)}\right]}\right\}\,dv\,dw.$	(46)

Expanding the term in braces gives

$\left[{\cos(t_1v)\cos\left({\rho\sigma_1wt_1\over\sigma_2}\right)-\sin(t_1v)\sin\left({\rho\sigma_1w\over\sigma_2t_1}\right)}\right]$
$+i\left[{\sin(t_1v)\cos\left({\rho\sigma_1w\over \sigma_2t_1}\right)+\cos(t_1v)\sin\left({\rho\sigma_1wt_1\over \sigma_2}\right)}\right]$
$=\left[{\cos\left({\rho\sigma_1wt_1\over \sigma_2}\right)+i\sin\left({\rho\sigma_1wt_1\over \sigma_2}\right)}\right]$
$[\cos(t_1v)+i\sin(t_1v)]= \mathop{\rm exp}\nolimits \left({{i\rho\sigma_1w\over \sigma_2} t_1}\right)[\cos(t_1v) + i\sin(t_1v)].$	(47)

But $e^{-ax^2}\sin(bx)$ is Odd, so the integral over the sine term vanishes, and we are left with

$\phi(t_1,t_2) = N' \int_{-\infty}^\infty e^{it_2w}\mathop{\rm exp}\nolimits \le... ...ht]\mathop{\rm exp}\nolimits \left[{i\rho\sigma_1wt_1\over \sigma_2}\right]\,dw$
$\times \int_{-\infty}^\infty \mathop{\rm exp}\nolimits \left[{-{v^2\over 2{\sigma_1}^2(1-\rho^2)}}\right]\cos(t_1v)\,dv$
$= N'\int_{-\infty}^\infty \mathop{\rm exp}\nolimits \left[{iw\left({t_2+t_1\lef... ...)}\right]\mathop{\rm exp}\nolimits \left[{-{w^2\over 2{\sigma_2}^2}}\right]\,dw$
$\times \int_{-\infty}^\infty \mathop{\rm exp}\nolimits \left[{-{v^2\over 2{\sigma_1}^2(1-\rho^2)}}\right]\cos(t_1v)\,dv.\quad$	(48)

Now evaluate the Gaussian Integral

$$\int_{-\infty}^\infty e^{ikx}e^{-ax^2}\,dx = \int_{-\infty}^\infty e^{-ax^2}\cos(kx)\,dx$$

$$= \sqrt{\pi\over a}e^{-k^2/4a}$$ (49)

to obtain the explicit form of the Characteristic Function,

$\phi(t_1,t_2) = {e^{i(t_1\mu_1+t_2\mu_2)}\over 2\pi\sigma_1\sigma_2\sqrt{1-\rho... ...ft({t_2+\rho{\sigma_1\over\sigma_2}t_1}\right)^2 2{\sigma_2}^2}\right]}\right\}$
$\quad\times\left\{{\sigma_1\sqrt{2\pi(1-\rho^2)}}{\mathop{\rm exp}\nolimits \left[{-{\textstyle{1\over 4}}{t_1}^2 2{\sigma_1}^2(1-\rho^2)}\right]}\right\}$
$\quad = e^{i(t_1\mu_1+t_2\mu_2)}\mathop{\rm exp}\nolimits \{-{\textstyle{1\over... ...1\sigma_2t_1t_2+\rho^2{\sigma_1}^2{t_1}^2+(1-\rho^2){\sigma_1}^2{t_1}^2]\}\quad$
$\quad = \mathop{\rm exp}\nolimits [i(t_1\mu_1+t_2\mu_2) - {\textstyle{1\over 2}}({\sigma_1}^2{t_1}^2+2\rho\sigma_1\sigma_2t_1t_2+{\sigma_1}^2{t_1}^2)].\quad$	(50)

Let $z_1$ and $z_2$ be two independent Gaussian variables with Means $\mu_i = 0$ and ${\sigma_i}^2 = 1$ for $i=1$, 2. Then the variables $a_1$ and $a_2$ defined below are Gaussian bivariates with unit Variance and Cross-Correlation Coefficient $\rho$:

$$a_1 = \sqrt{1+\rho\over 2} \,z_1 + \sqrt{1-\rho\over 2} \,z_2$$ (51)

$$a_2 = \sqrt{1+\rho\over 2} \,z_1 - \sqrt{1-\rho\over 2} \,z_2.$$ (52)

The conditional distribution is

$$P(x_2\vert x_1) = {1\over \sigma_2\sqrt{2\pi(1-\rho^2)}}\mat... ...limits \left[{- { (x^2-\mu'^2)^2\over 2{\sigma_2'}^2}}\right],$$ (53)

where

$${\mu'}_2 \equiv \mu_2 + \rho{\sigma_2\over\sigma_1} (x_1-\mu_1)$$ (54)

$${\sigma'}_2 \equiv \sigma_2\sqrt{1-\rho_2}\,.$$ (55)

The marginal probability density is

$$P(x_2) = \int_{-\infty}^\infty P(x_1,x_2)\,dx_1$$

$$= {1\over\sigma_2\sqrt 2\pi}\mathop{\rm exp}\nolimits \left[{-{(x_2-\mu_2)^2\over 2{\sigma_2}^2}}\right].$$ (56)

See also Box-Muller Transformation, Gaussian Distribution, McMohan's Theorem, Normal Distribution

References

Abramowitz, M. and Stegun, C. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, pp. 936-937, 1972.

Spiegel, M. R. Theory and Problems of Probability and Statistics. New York: McGraw-Hill, p. 118, 1992.