Moments And Variance Properties

1. Core Concept Overview & Intuition

Moments are numerical summaries of a probability distribution. They describe where a distribution is centred, how widely it spreads, and how its shape leans or peaks. The first moment about the origin is the mean (expectation), the second central moment is the variance, and higher moments capture skewness and kurtosis. Because moments are defined through expectation, every moment is a weighted average: for a discrete random variable the weights are probabilities, and for a continuous random variable the weights are supplied by the probability density function.

Variance measures dispersion, that is, the average squared deviation of a random variable from its mean. Its properties are the working tools of the subject: variance is unchanged by adding a constant (a shift of origin), it is multiplied by the square of a constant when the variable is rescaled (a change of scale), and it combines additively for independent variables. These two invariance rules are exactly the algebraic justification for the shortcut and coding techniques used in computation, where data are re-expressed relative to an assumed mean and rescaled by a convenient unit before moments are calculated.

The computational theme of this unit is that raw moments about an arbitrary point can always be converted into central moments about the mean, and vice versa. Working with an assumed mean or a coded variable reduces arithmetic without changing the underlying moments, provided the conversion formulas are applied correctly.

2. Mathematical Formalization & Core Definitions

2.1 Random Variables and Probability Distributions (Prerequisite)

> A random variable $X$ is a real-valued function defined on a sample space $S$, that is $X:S\to\mathbb{R}$. Capital letters denote random variables and small letters denote their values.

> A random variable is discrete if it assumes only a finite or countably infinite number of values on the real line. Its probability distribution is the function $f(x)=P(X=x)$ satisfying $f(x)\ge 0$ for all $x$ and $\sum_{x=-\infty}^{\infty}f(x)=1$.

> A random variable is continuous if it can assume any value in an interval of the real line. A continuous real-valued function $f(x)$ is a probability density function of $X$ if $f(x)\ge 0$ for all real $x$ and $\int_{-\infty}^{\infty}f(x)\,dx=1$, with $P(a\le X\le b)=\int_{a}^{b}f(x)\,dx$.

2.2 Expectation (Prerequisite)

> The expectation of a discrete random variable $X$ with probability distribution $f(x)$ is

$$
\mathbb{E}(X)=\sum_{x}x\,f(x),
$$

> and for a continuous random variable with density $f(x)$ it is

$$
\mathbb{E}(X)=\int_{-\infty}^{\infty}x\,f(x)\,dx.
$$

Expectation is linear: for constants $a$ and $b$, $\mathbb{E}(aX+b)=a\,\mathbb{E}(X)+b$.

2.3 Moments

> The $r$-th moment of a random variable $X$ about the origin (the $r$-th raw moment) is

$$
\mu_r'=\mathbb{E}(X^{r}),
$$

> so that $\mu_1'=\mathbb{E}(X)=\mu$ is the mean.

> The $r$-th central moment of $X$ about its mean is

$$
\mu_r=\mathbb{E}\left[(X-\mu)^{r}\right].
$$

The first central moment is always zero, $\mu_1=0$. The second central moment is the variance, $\mu_2=\operatorname{Var}(X)$. The third and fourth central moments, standardised by appropriate powers of the standard deviation, describe skewness and kurtosis respectively.

For a discrete random variable the moments take the summation form

$$
\mu_r'=\sum_{x}x^{r}f(x),\qquad \mu_r=\sum_{x}(x-\mu)^{r}f(x),
$$

and for a continuous random variable the integral form

$$
\mu_r'=\int_{-\infty}^{\infty}x^{r}f(x)\,dx,\qquad \mu_r=\int_{-\infty}^{\infty}(x-\mu)^{r}f(x)\,dx.
$$

2.4 Variance

> The variance of a random variable $X$ is the second central moment,

$$
\operatorname{Var}(X)=\mathbb{E}\left[(X-\mu)^{2}\right],
$$

> and the standard deviation is $\sigma=\sqrt{\operatorname{Var}(X)}$.

2.5 Shift of Origin (Assumed Mean)

Let $d=x-A$, where $A$ is an arbitrary assumed mean. The moments about $A$ are

$$
m_r'=\mathbb{E}\left[(X-A)^{r}\right].
$$

The raw moments about the origin are recovered from the moments about $A$ by the binomial expansion

$$
\mu_r'=\mathbb{E}\left[(X-A+A)^{r}\right]=\sum_{k=0}^{r}\binom{r}{k}A^{\,r-k}m_k',
$$

where $m_0'=1$. In particular,

$$
\mu_1'=m_1'+A,
$$

and the central moments are recovered from the moments about $A$ by

$$
\mu_r=\mathbb{E}\left[(X-A-( \mu-A))^{r}\right]=\sum_{k=0}^{r}\binom{r}{k}(-1)^{k}\,(\mu-A)^{k}\,m_{r-k}'.
$$

For the second central moment this gives the standard conversion

$$
\mu_2=m_2'-\left(m_1'\right)^{2},
$$

which is the computational form of the variance in terms of moments about any point $A$.

2.6 Change of Scale (Coding)

Let $u=\dfrac{x-A}{h}$, where $A$ is an assumed mean and $h$ is a constant class width or scale factor. Then $x=A+hu$, and the moments of $X$ are obtained from those of $U$ by

$$
\mu_r'=\mathbb{E}\left[(A+hU)^{r}\right]=\sum_{k=0}^{r}\binom{r}{k}A^{\,r-k}h^{k}\,\mathbb{E}(U^{k}).
$$

Because a shift of origin does not affect central moments and a change of scale multiplies them by the $r$-th power of the scale factor,

$$
\operatorname{Var}(X)=h^{2}\operatorname{Var}(U),
$$

and more generally $\mu_r(X)=h^{r}\mu_r(U)$ for the $r$-th central moment.

2.7 Grouped and Ungrouped Data

For ungrouped observations $x_1,x_2,\dots,x_n$ the raw and central moments are the arithmetic averages

$$
m_r'=\frac{1}{n}\sum_{i=1}^{n}x_i^{r},\qquad m_r=\frac{1}{n}\sum_{i=1}^{n}(x_i-\bar{x})^{r},
$$

where $\bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_i$. For grouped data with class midpoints $x_i$ and frequencies $f_i$, $i=1,\dots,k$, with $N=\sum_{i=1}^{k}f_i$, the corresponding formulas are

$$
m_r'=\frac{1}{N}\sum_{i=1}^{k}f_i x_i^{r},\qquad m_r=\frac{1}{N}\sum_{i=1}^{k}f_i(x_i-\bar{x})^{r}.
$$

With an assumed mean $A$ and coding $u_i=\dfrac{x_i-A}{h}$, the grouped moments about $A$ are

$$
m_r'=\frac{1}{N}\sum_{i=1}^{k}f_i\,(x_i-A)^{r}=h^{r}\cdot\frac{1}{N}\sum_{i=1}^{k}f_i u_i^{r},
$$

and the variance follows from $\operatorname{Var}(X)=h^{2}\left[\dfrac{1}{N}\sum_{i=1}^{k}f_i u_i^{2}-\left(\dfrac{1}{N}\sum_{i=1}^{k}f_i u_i\right)^{2}\right]$.

2.8 Geometric and Harmonic Means

> The geometric mean of $n$ positive observations is

$$
G=\left(\prod_{i=1}^{n}x_i\right)^{1/n},
$$

> equivalently $\log G=\dfrac{1}{n}\sum_{i=1}^{n}\log x_i$.

> The harmonic mean of $n$ positive observations is

$$
H=\frac{n}{\sum_{i=1}^{n}\dfrac{1}{x_i}}.
$$

For grouped data with frequencies $f_i$ attached to positive values $x_i$ and $N=\sum_{i=1}^{k}f_i$,

$$
G=\left(\prod_{i=1}^{k}x_i^{f_i}\right)^{1/N},\qquad H=\frac{N}{\sum_{i=1}^{k}\dfrac{f_i}{x_i}}.
$$

For positive data the three means satisfy the ordering $H\le G\le \bar{x}$, with equality throughout only when all observations are equal.