Moment-Generating Function

Given a Random Variable $x \in R$, if there exists an $h > 0$ such that

$$M(t) \equiv \left\langle{e^{tx}}\right\rangle{} = \cases{ \s... ...fty}^\infty e^{tx}P(x)\,dx & for a continuous distribution\cr}$$ (1)

for $\vert t\vert < h$, then

$$M(t) \equiv \langle e^{tx}\rangle$$ (2)

is the moment-generating function.

$$M(t) = \int_{-\infty}^\infty (1+tx+{\textstyle{1\over 2!}}t^2x^2+\ldots)P(x)\,dx$$

$$= 1+tm_1+{\textstyle{1\over 2!}}t^2 m_2+\ldots,$$ (3)

where $m_r$ is the $r$th Moment about zero. The moment-generating function satisfies

$$M_{x+y}(t) = \langle e^{t(x+y)}\rangle = \langle e^{tx}e^{ty}\rangle$$

$$= \langle e^{tx}\rangle \langle e^{ty}\rangle = M_x(t)M_y(t).$$ (4)

If $M(t)$ is differentiable at zero, then the $n$th Moments about the Origin are given by $M^{n}(0)$

$$M(t) = \langle e^{tx}\rangle \qquad M(0) = 1$$ (5)

$$M'(t) = \langle xe^{tx}\rangle \qquad M'(0) = \langle x\rangle$$ (6)

$$M''(t) = \langle x^2e^{tx}\rangle \qquad M''(0) = \langle x^2\rangle$$ (7)

$$M^{(n)}(t) = \langle x^ne^{tx}\rangle \qquad M^{(n)}(0)=\langle x^n\rangle.$$ (8)

The Mean and Variance are therefore

$$\mu \equiv \langle x\rangle = M'(0)$$ (9)

$$\sigma^2 \equiv \langle x^2\rangle -\langle x\rangle^2 = M''(0)-[M'(0)]^2.$$ (10)

It is also true that

$$\mu_n=\sum_{j=0}^n{n\choose j}(-1)^{n-j}\mu_j'(\mu'_1)^{n-j},$$ (11)

where $\mu'_0=1$ and $\mu_j'$ is the $j$th moment about the origin.

It is sometimes simpler to work with the Logarithm of the moment-generating function, which is also called the Cumulant-Generating Function, and is defined by

$$R(t) \equiv \ln[M(t)]$$ (12)

$$R'(t) = {M'(t)\over M(t)}$$ (13)

$$R''(t) = {M(t)M''(t)-[M'(t)]^2\over[M(t)]^2}.$$ (14)

But $M(0)=\left\langle{1}\right\rangle{}=1$, so

$$\mu = M'(0) = R'(0)$$ (15)

$$\sigma^2 = M''(0)-[M'(0)]^2 = R''(0).$$ (16)

See also Characteristic Function, Cumulant, Cumulant-Generating Function, Moment

References

Kenney, J. F. and Keeping, E. S. ``Moment-Generating and Characteristic Functions,'' ``Some Examples of Moment-Generating Functions,'' and ``Uniqueness Theorem for Characteristic Functions.'' §4.6-4.8 in Mathematics of Statistics, Pt. 2, 2nd ed. Princeton, NJ: Van Nostrand, pp. 72-77, 1951.