Moments

1. Core Concept Overview & Intuition

The theory of moments is the quantitative backbone of probability. A random variable $X$ is a numerical summary of the outcome of a random experiment, and its distribution is described not by a single number but by a whole family of averages called moments. The first moment about zero, $E(X)$, is the long-run average or centre of gravity of the distribution. Higher moments measure spread, asymmetry, and tail weight. The absolute moments $E\lvert X\rvert^{r}$ control how much probability can sit far from the origin, and this control is precisely what converts moment information into probability bounds and convergence statements.

The central intuition is an inequality principle: if the average size of a quantity is small, then the probability that the quantity is large must also be small. This single idea, formalised in the basic inequality, yields Markov's inequality, Chebyshev's inequality, and sufficient conditions for convergence in probability. A second intuition is convexity: averaging a convex function of $X$ can only increase the value relative to applying the function to the average, which is Jensen's inequality. A third intuition is that the family of moments is nested: existence of a high-order moment forces existence of every lower-order moment, so the moment hierarchy is a measure of how heavy the tails of a distribution are.

Throughout, $X$ and $Y$ denote random variables on a probability space with sample space $S$ and probability measure $P$; $E(\cdot)$ denotes expectation; $r$ and $s$ are positive real exponents; $c$ is a real constant; $\mu=E(X)$ is the mean and $\sigma^{2}=E\bigl((X-\mu)^{2}\bigr)$ is the variance.

2. Mathematical Formalization & Core Definitions

2.1 Expectation via indicator partitions

Let $X$ be a discrete random variable taking values $x_{1},\dots,x_{n}$ on the events $A_{i}=[X=x_{i}]$, which form a partition of $S$. Writing $I_{A_{i}}$ for the indicator of $A_{i}$, so that $X=\sum_{i=1}^{n}x_{i}I_{A_{i}}$, the expectation is defined by

$$
E(X)=E!\left(\sum_{i=1}^{n}x_{i}I_{A_{i}}\right)=\sum_{i=1}^{n}x_{i}E(I_{A_{i}})=\sum_{i=1}^{n}x_{i}P(A_{i})=\sum_{i=1}^{n}x_{i}p_{i},
$$

or equivalently

$$
E(X)=\sum_{i=1}^{n}x_{i}P(X=x_{i}).
$$

Here $p_{i}=P(X=x_{i})$ and $\sum_{i=1}^{n}p_{i}=1$. The key step is $E(I_{A_{i}})=P(A_{i})$, which follows because an indicator takes only the values $0$ and $1$.

2.2 Properties of expectation

> Linearity. For random variables $X$ and $Y$, $E(X\pm Y)=E(X)\pm E(Y)$.

The proof writes $X=\sum_{i=1}^{m}x_{i}I_{A_{i}}$ and $Y=\sum_{j=1}^{n}y_{j}I_{B_{j}}$ over partitions ${A_{i}}$ and ${B_{j}}$ of $S$. The product indicators satisfy $I_{A_{i}}I_{B_{j}}=I_{A_{i}\cap B_{j}}$, so

$$
X\pm Y=\sum_{i=1}^{m}\sum_{j=1}^{n}(x_{i}\pm y_{j})I_{A_{i}\cap B_{j}}.
$$

Taking expectations and splitting the sum,

$$
E(X\pm Y)=\sum_{i=1}^{m}\sum_{j=1}^{n}x_{i}P(A_{i}\cap B_{j})\pm\sum_{i=1}^{m}\sum_{j=1}^{n}y_{j}P(A_{i}\cap B_{j}).
$$

Because ${B_{j}}$ partitions $S$, $\sum_{j=1}^{n}P(A_{i}\cap B_{j})=P(A_{i})$, and because ${A_{i}}$ partitions $S$, $\sum_{i=1}^{m}P(A_{i}\cap B_{j})=P(B_{j})$. Hence

$$
E(X\pm Y)=\sum_{i=1}^{m}x_{i}P(A_{i})\pm\sum_{j=1}^{n}y_{j}P(B_{j})=E(X)\pm E(Y).
$$

> Scale preserving. For any constant $c$, $E(cX)=cE(X)$.

> Non-negativity. If $X\ge 0$ then $E(X)\ge 0$.

2.3 Moments about zero and about a point

> Definition (Moments). Let $X$ be a random variable. The $r$th moment of $X$ about zero is $E(X^{r})$; the $r$th absolute moment about zero is $E\lvert X\rvert^{r}$; and the $r$th absolute moment of $X$ about a point $a$ is $E\lvert X-a\rvert^{r}$.

The absolute moment about the mean, $E\lvert X-\mu\rvert^{r}$, is the natural measure of dispersion of order $r$; for $r=2$ it reduces to the variance $\sigma^{2}$.

2.4 The moment hierarchy

> Lemma (Nested moments). If $E\lvert X\rvert^{r}<\infty$ and $E\lvert X\rvert^{s}<\infty$ with $0<s<r$, then $E\lvert X\rvert^{s}\le 1+E\lvert X\rvert^{r}$.

The proof splits on the size of $\lvert X\rvert$. If $\lvert X\rvert\le 1$ then $\lvert X\rvert^{s}\le 1$; if $\lvert X\rvert\ge 1$ then, since $s<r$, $\lvert X\rvert^{s}\le\lvert X\rvert^{r}$. Therefore for every outcome

$$
\lvert X\rvert^{s}\le 1+\lvert X\rvert^{r},\qquad 0<s<r.
$$

Taking expectations and using monotonicity of expectation gives $E\lvert X\rvert^{s}\le 1+E\lvert X\rvert^{r}<\infty$. The practical consequence is that a finite moment of order $r$ guarantees finiteness of every lower-order absolute moment.

2.5 Convex functions

> Definition (Convexity). Let $f$ be a real-valued Borel function on an open interval $I\subseteq\mathbb{R}$ and let $x,x'\in I$. Then $f$ is convex if

$$
f!\left(\frac{x+x'}{2}\right)\le \frac{1}{2}f(x)+\frac{1}{2}f(x').
$$

> Equivalently, for each $x_{0}\in I$ there is a number $\lambda(x_{0})$ such that

$$
\frac{f(x)-f(x_{0})}{x-x_{0}}\ge\lambda(x_{0}),\qquad x\ne x_{0},
$$

> that is, the graph of $f$ lies above the line through $(x_{0},f(x_{0}))$ with slope $\lambda(x_{0})$:

$$
f(x)\ge f(x_{0})+\lambda(x_{0})(x-x_{0}).
$$

The slope $\lambda(x_{0})$ is a supporting-line slope at $x_{0}$; for differentiable $f$ it is $f'(x_{0})$. The geometric content is that a convex curve never lies below any of its tangent or supporting lines.