Expectation

1. Core Concept Overview & Intuition

Expectation is the probability-weighted average value of a random variable. Rather than averaging observed data, expectation averages every possible value of the variable, weighting each value by the probability (discrete case) or probability density (continuous case) attached to it. It is the theoretical counterpart of the arithmetic mean and is the first building block for every later summary measure of a distribution.

The central idea is a weighted sum or weighted integral. For a discrete random variable, each value $x$ contributes $x$ multiplied by its point probability $f(x)=P(X=x)$. For a continuous random variable, the discrete sum is replaced by an integral of $x$ against the density $f(x)$. In both cases the operation is linear: constants factor out and sums of functions split into sums of expectations. This linearity is the single most useful computational property and is used repeatedly to evaluate expectations of polynomial and affine functions of a random variable.

A second idea is that expectation applies not only to $X$ but to any function $g(X)$ of $X$. The value $g(x)$ is weighted by the same probability or density as $x$, so no separate distribution for $g(X)$ needs to be derived. This is what allows direct computation of quantities such as $E(X^2)$ or $E\big([X-E(X)]^2\big)$.

A third idea is existence. The expectation is defined through a sum or integral that may fail to converge. The notes record the principle that if $E(X)$ exists then $E\lvert X\rvert$ also exists, and if $E\lvert X\rvert$ does not exist then $E\lvert X\rvert=\infty$. A distribution can therefore have no finite mean even though its density is perfectly well defined everywhere.

2. Mathematical Formalization & Core Definitions

> Definition (Expectation of a discrete random variable). Let $X$ be a discrete random variable whose probability distribution is defined by $f(x)=P(X=x)$. The expected value of $X$, denoted $E(X)$, is defined by

$$
E(X)=\sum_{x=-\infty}^{\infty}xf(x).
$$

> Definition (Expectation of a function of a discrete random variable). Let $g(X)$ be a new random variable depending on a discrete random variable $X$. Then

$$
E[g(X)]=\sum_{x=-\infty}^{\infty}g(x)f(x).
$$

> Definition (Expectation of a continuous random variable). Let $X$ be a continuous random variable with probability density function $f(x)$. Then the expected value of $X$ is defined by the integral

$$
E(X)=\int_{-\infty}^{\infty}xf(x)\,dx.
$$

> Definition (Expectation of a function of a continuous random variable). More generally, if $g(X)$ is a function of $X$, then

$$
E[g(X)]=\int_{-\infty}^{\infty}g(x)f(x)\,dx.
$$

> Existence principle. If $E(X)$ exists then $E\lvert X\rvert$ also exists. If $E\lvert X\rvert$ does not exist, then $E\lvert X\rvert=\infty$.

The definitions are applied identically to ungrouped and grouped descriptions of a discrete distribution: in every case the value of the variable is multiplied by its probability and the products are summed over all support points. For a distribution uniform on the integers $1,2,\dots,n$, each point carries probability $1/n$, and the sum reduces to the arithmetic mean of the support values. For a distribution uniform on an interval $(a,b)$, the density is the constant $1/(b-a)$ and the integral reduces to the midpoint of the interval. These are the two canonical benchmarks against which other expectations are compared.