Random Variables

1. Core Concept Overview & Intuition

A statistical experiment produces outcomes that cannot be predicted with certainty in advance. The set of all possible outcomes of such an experiment is called the sample space, denoted $S$. Working directly with the raw outcomes of $S$ is often awkward: outcomes may be words, symbols, or complicated descriptions rather than numbers. A random variable is the device that converts each outcome into a real number, so that the full machinery of real analysis and algebra can be applied to the experiment.

The central idea is that a random variable is a rule of assignment, not a number. The rule is fixed before the experiment is performed, but the numerical value it produces depends on which outcome actually occurs. Capital letters such as $X$, $Y$, $Z$ denote random variables, while small letters such as $x$, $y$, $z$ denote the particular values the random variable may take. This notational convention is essential: $X$ is the function, and $x$ is a point in its range.

Once a random variable is defined, the next task is to attach probabilities to its possible values. This attachment is the probability distribution of the random variable. The nature of the distribution depends on whether the random variable takes isolated values or fills an interval. A random variable that assumes only a finite or countable number of values on the real line is discrete; a random variable that can assume any value in an interval of the real line is continuous. Discrete random variables are described by a probability mass function, which assigns a positive probability to each individual value, and these probabilities must sum to one. Continuous random variables are described by a probability density function, which is integrated over intervals to produce probabilities, and the total area under the density must equal one.

The distinction has a practical consequence for how probabilities are computed. For a discrete random variable, the probability of a value is read directly from the distribution. For a continuous random variable, the probability of any single point is zero, and probabilities are obtained only by integrating the density over an interval. The probability that a continuous random variable lies between two limits $a$ and $b$ is the area under the density curve between those limits.

A recurring theme in the worked material is the determination of an unknown constant in a proposed distribution. Because a probability distribution must total one, the unknown constant is fixed by imposing the normalization condition: a sum equal to one in the discrete case, or an integral equal to one in the continuous case. This single principle underlies almost every calculation in this topic.

2. Mathematical Formalization & Core Definitions

> Definition (Random Variable). Let $S$ be a sample space representing the outcomes of a statistical experiment. A random variable $X$ is a real-valued function defined on $S$, that is

$$
X: S \to \mathbb{R}.
$$

> Capital letters denote random variables and small letters denote the respective values of the random variables.

The definition is illustrated by a selection experiment. Suppose three boys are selected at random from a school parade and each is asked whether he smokes, denoted $S$, or does not, denoted $N$. The sample space is

$$
S={SSS,\,SSN,\,SNS,\,NSS,\,SNN,\,NSN,\,NNS,\,NNN}.
$$

Let $X$ denote the number of smokers among the three chosen boys. Then the assignment rule gives

$$
X(SSS)=3,\qquad X(SSN)=X(SNS)=X(NSS)=2,
$$

$$
X(SNN)=X(NSN)=X(NNS)=1,\qquad X(NNN)=0.
$$

Thus $X$ is a random variable taking the values $0,1,2,3$.

A second illustration shows that a random variable need not be discrete. Suppose a real number is selected at random in the closed interval $[0,2]$, and let $X$ denote the number so chosen. Then $X$ is a random variable with possible values $x$ satisfying $0\le x\le 2$.

2.1 Discrete Random Variables

> Definition (Discrete Random Variable). A random variable is said to be discrete if it assumes only a finite or countable number of values on the real line.

A discrete random variable assumes each of its values with a certain probability. In the three-boys experiment, if all eight outcomes are assumed equally likely, then $X$ takes the values $0,1,2,3$ with the following probabilities:

$$
P(X=0)=P{NNN}=\frac{1}{8},
$$

$$
P(X=1)=P{SNN,\,NSN,\,NNS}=\frac{3}{8},
$$

$$
P(X=2)=P{SSN,\,SNS,\,NSS}=\frac{3}{8},
$$

$$
P(X=3)=P{SSS}=\frac{1}{8}.
$$

These probabilities are displayed in tabular form as follows.

The probabilities satisfy

$$
P(X=0)+P(X=1)+P(X=2)+P(X=3)=1.
$$

Such a table represents a probability distribution of the random variable $X$.

> Definition (Probability Distribution of a Discrete Random Variable). Let $X$ be a discrete random variable. The probability distribution of $X$ is a real-valued function $f(x)$ of $x$, defined by

$$
f(x)=P(X=x),
$$

> and satisfying the following conditions:

> (i) $f(x)\ge 0$ for all $x$;

> (ii) $\displaystyle\sum_{x=-\infty}^{\infty} f(x)=1$.

The two conditions are the defining test for any proposed discrete distribution: non-negativity everywhere, and total mass one.

> Definition (Discrete Uniform Distribution). A probability distribution given by

$$
f(x)=\begin{cases}\dfrac{1}{n}, & x=1,2,\dots,n,
0, & \text{otherwise},\end{cases}
$$

> is known as a discrete uniform distribution.

Example (digit selected at random). A digit is selected at random from among the digits $0,1,2,3,4,5,6,7$, and $X$ denotes the digit so selected. Since there are eight equally likely digits,

$$
P(X=x)=\frac{1}{8},\qquad x=0,1,2,3,4,5,6,7,
$$

and $P(X=x)=0$ for $x\notin{0,1,2,3,4,5,6,7}$. Therefore the probability distribution of $X$ is

$$
f(x)=\begin{cases}\dfrac{1}{8}, & x=0,1,2,3,4,5,6,7,
0, & \text{otherwise}.\end{cases}
$$

Clearly $f(x)\ge 0$ and $\displaystyle\sum_{x=0}^{7} f(x)=1$.

Example (an infinite discrete distribution). Let $X$ be a discrete random variable whose set of values is the set of all non-negative integers. The function

$$
f(x)=\begin{cases}\dfrac{1}{2^{x+1}}, & x=0,1,2,\dots,
0, & \text{otherwise},\end{cases}
$$

is a probability distribution. Indeed $f(x)\ge 0$ for all $x$, and the total mass is the geometric series

$$
\sum_{x=0}^{\infty} f(x)=\sum_{x=0}^{\infty}\frac{1}{2^{x+1}}=\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\cdots=1.
$$

Example (determining a constant by normalization). Let $X$ be a discrete random variable whose probability distribution is

$$
f(x)=\begin{cases}kx, & x=2,3,4,5,6,
0, & \text{otherwise}.\end{cases}
$$

Since $f(x)$ is a probability distribution, the total mass must equal one:

$$
\sum_{x=2}^{6} f(x)=1.
$$

Substituting the proposed form gives

$$
\sum_{x=2}^{6} kx=1,
$$

so that

$$
2k+3k+4k+5k+6k=1,
$$

hence

$$
20k=1,
$$

and therefore

$$
k=\frac{1}{20}.
$$

This example displays the standard technique for every constant-finding problem in the discrete case: write the normalization sum, substitute the proposed mass function, collect the constant, and solve.

2.2 Continuous Random Variables

> Definition (Continuous Random Variable). A random variable is said to be continuous if it can assume any value in an interval of the real line $\mathbb{R}$. A continuous random variable therefore assumes an uncountable number of values.

> Definition (Probability Density Function). Let $X$ be a continuous random variable assuming values in $\mathbb{R}$. A continuous real-valued function $f(x)$ is said to be a probability density function (p.d.f.), or simply a probability distribution, of the random variable $X$ if it satisfies the following conditions:

> (i) $f(x)\ge 0$ for all real $x$;

> (ii) $\displaystyle\int_{-\infty}^{\infty} f(x)\,dx=1$.

For a continuous random variable, probabilities are obtained by integration over intervals:

$$
P(a\le X\le b)=\int_{a}^{b} f(x)\,dx.
$$

The density itself is not a probability; only its integral over an interval carries probabilistic meaning.

Example (verifying a p.d.f. and computing probabilities). Let $X$ be a continuous random variable with proposed density

$$
f(x)=\begin{cases}\dfrac{1}{2}x, & 0\le x\le 2,
0, & \text{otherwise}.\end{cases}
$$

Clearly $f(x)\ge 0$ for all real $x$. The normalization integral is

$$
\int_{0}^{2} f(x)\,dx=\int_{0}^{2}\frac{1}{2}x\,dx=1,
$$

so $f(x)$ is a p.d.f. of $X$. The required probabilities are then computed by integrating the density over the stated intervals:

$$
P\left(\frac{1}{2}\le X\le 1\right)=\int_{1/2}^{1}\frac{1}{2}x\,dx=\frac{3}{16},
$$

and

$$
P(-1\le X\le 1)=\int_{-1}^{0} f(x)\,dx+\int_{0}^{1} f(x)\,dx=0+\frac{1}{4}=\frac{1}{4}.
$$

The first integral on the left is zero because the density vanishes outside $[0,2]$. This illustrates the general rule that the density contributes nothing over regions where it is defined to be zero.

Example (constant determined by integration). Let $X$ be a continuous random variable whose p.d.f. is

$$
f(x)=\begin{cases}kx+\dfrac{1}{30}, & 0\le x\le 3,
0, & \text{elsewhere},\end{cases}
$$

where $k$ is a constant. Since $f(x)$ is a p.d.f.,

$$
\int_{0}^{3} f(x)\,dx=1.
$$

Substituting the proposed density gives

$$
\int_{0}^{3}\left(kx+\frac{1}{30}\right)dx=1,
$$

which yields

$$
k=\frac{1}{5}.
$$

Therefore

$$
f(x)=\begin{cases}\dfrac{1}{5}x+\dfrac{1}{30}, & 0\le x\le 3,
0, & \text{elsewhere},\end{cases}
$$

and the required probability is

$$
P(1\le X\le 2)=\int_{1}^{2}\left(\frac{1}{5}x+\frac{1}{30}\right)dx=\frac{1}{3}.
$$

Example (rectangular density). Let

$$
f(x)=\begin{cases}c, & a<x<b,
0, & \text{otherwise},\end{cases}
$$

be a p.d.f. of a continuous random variable $X$. Since $f(x)$ is a p.d.f.,

$$
\int_{a}^{b} f(x)\,dx=1,
$$

so that

$$
\int_{a}^{b} c\,dx=1,
$$

and hence

$$
c=\frac{1}{b-a}.
$$

Therefore

$$
f(x)=\begin{cases}\dfrac{1}{b-a}, & a<x<b,
0, & \text{otherwise}.\end{cases}
$$

Such a distribution is known as a rectangular density function on the interval $(a,b)$. If $a=0$ and $b=1$, then

$$
f(x)=\begin{cases}1, & 0<x<1,
0, & \text{otherwise},\end{cases}
$$

which is the uniform density function on the unit interval $(0,1)$.