The Pgf And Moments Of Standard Distributions

1. Core Concept Overview & Intuition

The probability generating function (p.g.f.) is a transform that encodes an entire probability distribution of a non-negative integer-valued random variable into a single analytic function of an auxiliary variable $s$. Once this function is known, the moments of the distribution — mean, variance, and higher factorial moments — can be extracted by differentiation at $s=1$, without ever returning to the original probability mass function. This is the central economy of the method: one function, differentiated, yields all moments.

The intuition rests on a simple observation. If $X$ takes values $k=0,1,2,\dots$ with probabilities $p_k$, then the function

$$
P(s)=\sum_{k=0}^{\infty}p_k s^k
$$

is a power series whose coefficient of $s^k$ is exactly $p_k$. Differentiation of a power series brings down the exponent as a multiplier, so the $r$-th derivative at $s=1$ produces the $r$-th factorial moment $\mathbb{E}[X(X-1)\cdots(X-r+1)]$. The mean and variance then follow by combining the first two factorial moments. The p.g.f. therefore acts as a compact "moment factory."

A second intuition concerns the relationship between the p.g.f. and the moment generating function (m.g.f.). Substituting $s=e^{t}$ converts the p.g.f. into the m.g.f., so the two transforms are the same object viewed through different coordinates. This link allows results proved for one to be transferred to the other.

The topic also develops two structural properties that make the p.g.f. indispensable in later probability modelling. First, the p.g.f. of a linear transform $Y=mX+n$ is obtained by a simple substitution and scaling, $P_Y(s)=s^{n}P(s^{m})$. Second, the p.g.f. of a sum of independent random variables is the product of their individual p.g.f.s. These two properties, together with the moment formulas, constitute the working toolkit for standard distributions.

The standard distributions treated here — Bernoulli, binomial, Poisson, and geometric — each have a p.g.f. that can be derived in closed form by summing a geometric or exponential series. From each closed form, the mean and variance are recovered by the same three-step routine: differentiate once, differentiate twice, then apply the variance identity. The derivations are mechanical and repeatable, which is precisely why they are examinable.

2. Mathematical Formalization & Core Definitions

> Definition (Probability Generating Function). Let $X$ be a random variable taking values in ${0,1,2,\dots}$ with probability mass function $p_k=\Pr(X=k)$. The probability generating function of $X$ is the function

$$
P(s)=\mathbb{E}[s^{X}]=\sum_{k=0}^{\infty}p_k s^k,
$$

> defined for all real $s$ for which the series converges absolutely.

The variable $s$ is a dummy argument; it carries no probabilistic meaning of its own. The requirement that $X$ be non-negative integer-valued is essential, because only then does $s^{X}$ make sense as a power of $s$ with integer exponent.

> Definition (Factorial Moments via the p.g.f.). For a random variable $X$ with p.g.f. $P(s)$, the factorial moments are obtained by differentiation at $s=1$:

$$
\mathbb{E}[X(X-1)]=P''(1),\qquad \mathbb{E}[X]=P'(1).
$$

The first factorial moment $\mathbb{E}[X(X-1)]$ is not the second raw moment; it is the second factorial moment. The raw second moment is recovered by adding back the mean, since $X^2=X(X-1)+X$.

> Definition (Moment Generating Function Link). When $s=e^{t}$, the p.g.f. becomes

$$
P(e^{t})=\mathbb{E}[e^{tX}],
$$

> which is the moment generating function of $X$.

This identity is the bridge between the two transform theories. It is valid wherever both sides converge.

> Definition (Integral of the p.g.f.). Integrating the p.g.f. over $s\in[0,1]$ term by term gives

$$
\int_0^1 P(s)\,ds=\sum_{k=0}^{\infty}\frac{p_k}{k+1}=\mathbb{E}!\left[\frac{1}{X+1}\right].
$$

This result is a by-product of the power-series structure and is occasionally useful for computing expectations of reciprocals.

> Definition (Recovery of Probabilities). The individual probabilities are recovered from the p.g.f. by either of two equivalent routes:

$$
p_k=\frac{1}{k!}\left.\frac{d^k P(s)}{ds^k}\right|_{s=0},
$$

> or, equivalently, $p_k$ is the coefficient of $s^k$ in the power-series expansion of $P(s)$ about $s=0$.

The second route is usually faster in practice: expand the closed form and read off the coefficient. For example, expanding $P(s)=e^{-\lambda(1-s)}=e^{-\lambda}e^{\lambda s}$ gives coefficient of $s^k$ equal to $e^{-\lambda}\lambda^k/k!$, identifying the Poisson distribution with parameter $\lambda$.

> Definition (p.g.f. of a Linear Transform). If $X$ has p.g.f. $P(s)$ and $Y=mX+n$ with integers $m\neq 0$ and $n$, then the p.g.f. of $Y$ is

$$
P_Y(s)=s^{n}P(s^{m}).
$$

The derivation is a direct computation using the definition and the laws of exponents: $P_Y(s)=\mathbb{E}[s^{mX+n}]=s^{n}\mathbb{E}[(s^{m})^{X}]=s^{n}P(s^{m})$.