Moments

1. Core Concept Overview & Intuition

A moment is a summary number that describes a feature of a probability distribution or a frequency distribution. The word "moment" is borrowed from mechanics, where moments describe how mass is distributed about a point. In statistics, the "mass" is probability or frequency, and the reference point is usually the arithmetic mean.

Moments taken about the mean are called central moments. They are the natural language for describing the shape of a distribution, because they are measured relative to the centre and are therefore unaffected by where the distribution is located on the number line. A distribution shifted bodily to the left or right keeps the same central moments; only its location changes.

The central moments are indexed by an integer order $r=1,2,3,4,\dots$ and each order captures a distinct geometric feature:

• The first central moment measures deviation about the mean and is always zero. It carries no shape information; it merely confirms that the mean is the balance point.
• The second central moment measures how far observations spread out from the mean. It is the variance, the fundamental measure of dispersion.
• The third central moment measures whether the distribution leans to one side. It is the raw material of skewness, the measure of asymmetry.
• The fourth central moment measures how much weight lies in the tails and at the peak. It is the raw material of kurtosis, the measure of peakedness.

The intuition is cumulative: the second moment tells you how wide the distribution is, the third tells you whether it is lopsided, and the fourth tells you whether it is sharply peaked with heavy tails or flat with light tails. Together, the second, third, and fourth central moments give a compact numerical fingerprint of a distribution's shape.

A useful mental picture is a histogram drawn on an axis. The mean is the point on the horizontal axis where the histogram would balance. The variance is the average squared horizontal distance of the bars from that balance point. The third moment asks whether the long tail of the histogram points right or left. The fourth moment asks whether the histogram has a tall narrow spike with long thin tails, or a low broad hump with short tails.

2. Mathematical Formalization & Core Definitions

> Definition (Central Moment). Let $X$ be a variable with mean $\mu$ and let $f_i$ denote the frequency of the value $x_i$ for $i=1,2,\dots,n$, with total frequency $N=\sum_{i=1}^{n} f_i$. The $r$-th central moment about the mean, denoted $m_r$, is the frequency-weighted average of the $r$-th powers of the deviations from the mean:

$$
m_r=\frac{1}{N}\sum_{i=1}^{n} f_i\,(x_i-\mu)^r.
$$

> For a probability distribution, the frequency weights are replaced by probabilities and the sum becomes an expectation.

The symbol $\mu$ denotes the arithmetic mean, $x_i$ denotes the $i$-th observed value or class mark, $f_i$ denotes its frequency, $N$ denotes the total number of observations, and $r$ denotes the order of the moment. The first central moment is

$$
m_1=\frac{1}{N}\sum_{i=1}^{n} f_i\,(x_i-\mu)=0,
$$

because the deviations from the mean sum to zero. This is why shape description begins at the second moment.

> Definition (Second Central Moment: Variance). The second central moment is the variance:

$$
m_2=\frac{1}{N}\sum_{i=1}^{n} f_i\,(x_i-\mu)^2=\sigma^2.
$$

> Its positive square root $\sigma$ is the standard deviation.

The variance is expressed in squared units of the original variable, while the standard deviation is expressed in the original units. The variance is the average of squared deviations from the mean, exactly as stated in the course summary of measures of dispersion.

> Definition (Third Central Moment and Skewness). The third central moment is

$$
m_3=\frac{1}{N}\sum_{i=1}^{n} f_i\,(x_i-\mu)^3.
$$

> Skewness is the measure of asymmetry built from this moment. The moment coefficient of skewness is

$$
\beta_1=\frac{m_3^{\,2}}{m_2^{\,3}},
$$

> and the corresponding signed coefficient is

$$
\gamma_1=\frac{m_3}{m_2^{\,3/2}}.
$$

The sign of $m_3$ carries the direction of the asymmetry. If $m_3>0$ the distribution has a longer right tail and is positively skewed; if $m_3<0$ it has a longer left tail and is negatively skewed. For a perfectly symmetrical distribution such as the normal curve, $m_3=0$ and therefore $\beta_1=0$. Because $\beta_1$ is a square, it is always non-negative and reports only the strength of asymmetry, not its direction; $\gamma_1$ preserves the sign. Measures of coefficient of skewness are used mainly for making comparisons between two or more distributions, and one may speak of slight skewness, moderate skewness, or marked skewness.

> Definition (Fourth Central Moment and Kurtosis). The fourth central moment is

$$
m_4=\frac{1}{N}\sum_{i=1}^{n} f_i\,(x_i-\mu)^4.
$$

> Kurtosis is the measure of peakedness built from this moment. The moment coefficient of kurtosis is

$$
\beta_2=\frac{m_4}{m_2^{\,2}}.
$$

Because the fourth power is always non-negative, $m_4>0$ for any non-degenerate distribution, and $\beta_2$ is always positive. The value $\beta_2=3$ is the benchmark set by the normal distribution. A distribution with $\beta_2>3$ is leptokurtic: it is more sharply peaked with heavier tails than the normal curve. A distribution with $\beta_2<3$ is platykurtic: it is flatter with lighter tails than the normal curve. A distribution with $\beta_2=3$ is mesokurtic.

The following table summarises the correspondence between moment order and shape feature.