Change Of Variable

1. Core Concept Overview & Intuition

The change of variable technique answers a single, central question: given a random variable $X$ whose probability distribution is known, what is the probability distribution of a new random variable $U=\Phi(X)$, where $\Phi$ is a specified function of $X$? The function $\Phi$ transforms the original variable into a new one, and the task is to transfer the probability mass (in the discrete case) or the probability density (in the continuous case) from the $x$-axis onto the $u$-axis.

The intuition differs sharply between the discrete and continuous settings.

In the discrete case, probability is concentrated on isolated points. Each possible value $u$ of $U$ arises from the collection of all values $x$ of $X$ that satisfy $\Phi(x)=u$. Because these $x$-values are mutually exclusive events, the probability that $U$ equals $u$ is simply the sum of the probabilities of those $x$-values. The transformation may merge several distinct $x$-values into one $u$-value, so the new probability function is obtained by aggregation.

In the continuous case, probability is spread over intervals and individual points carry zero probability. One therefore cannot sum probabilities point by point. Instead, the standard route is to work with the cumulative distribution function: compute $G(u)=P(U\le u)$ by translating the event ${\Phi(X)\le u}$ into an equivalent event about $X$, integrate the density $f(x)$ over the resulting region, and then differentiate $G$ to recover the density $g(u)=G'(u)$. A second, more direct route exploits the inverse function: if $x=\omega(u)$ is the inverse of $\Phi$, the density of $U$ is obtained by substituting $x=\omega(u)$ into $f$ and multiplying by the absolute value of the derivative $\frac{d\omega(u)}{du}$. This factor accounts for the local stretching or compression of the $x$-axis as it is mapped onto the $u$-axis.

A useful mental picture is that of relabelling the horizontal axis. The total probability, represented by the area under the density curve, must remain equal to one after relabelling. When the transformation stretches an interval of $x$-values into a longer interval of $u$-values, the density must correspondingly become smaller; when it compresses, the density becomes larger. The derivative factor $\left\lvert\frac{d\omega(u)}{du}\right\rvert$ is precisely the measure of this stretching or compression.

2. Mathematical Formalization & Core Definitions

> Definition (Function of a random variable). Let $X$ be a random variable and let $\Phi$ be a real-valued function. Then $U=\Phi(X)$ is itself a random variable, called a function of the random variable $X$. The distribution of $U$ is called the distribution of the transformed variable.

> Definition (Probability function of a transformed discrete variable). Suppose $X$ has a discrete distribution with probability function $f(x)$. Let $U=\Phi(X)$. Then the probability function $g(u)$ of $U$ is defined by

$$
g(u)=P(U=u)=P(\Phi(X)=u)=\sum_{x:\,\Phi(x)=u} f(x).
$$

> The sum runs over every value $x$ in the support of $X$ that is mapped onto $u$ by $\Phi$.

> Definition (Distribution function of a transformed continuous variable). Suppose $X$ is a continuous random variable with probability density function $f(x)$, and let $U=\Phi(X)$. For any real number $u$, the cumulative distribution function $G(u)$ of $U$ is

$$
G(u)=P(U\le u)=P(\Phi(X)\le u)=\int_{x:\,\Phi(x)\le u} f(x)\,dx.
$$

> If $G$ is a continuous function of $u$, then at every point $u$ at which $G$ is differentiable, the probability density function of $U$ is

$$
g(u)=\frac{dG(u)}{du}.
$$

> Definition (Inverse transformation). Suppose $U=\Phi(X)$ with $a<x<b$ and $\alpha<u<\beta$. If $x=\omega(u)$ satisfies $\Phi(\omega(u))=u$, then $\omega$ is the inverse function of $\Phi$. When $\Phi$ is continuous and strictly monotone on $(a,b)$, the inverse $\omega$ is continuous and strictly monotone on $(\alpha,\beta)$.

> Definition (Standardized random variable). For a random variable $X$ with mean $\mu$ and variance $\sigma^{2}$, the standardized random variable is

$$
Z=\frac{X-\mu}{\sigma}.
$$

> It is a change of variable of the linear form $Z=\Phi(X)$ with $\Phi(x)=\frac{x-\mu}{\sigma}$, and it satisfies $E(Z)=0$ and $\operatorname{Var}(Z)=1$.

> Definition (Valid probability density function). A real-valued function $f(x)$ is a probability density function of a continuous random variable $X$ if and only if it satisfies both of the following conditions:

> 1. Non-negativity: $f(x)\ge 0$ for all real $x$.
> 2. Total probability equals one: $\displaystyle\int_{-\infty}^{\infty} f(x)\,dx=1$.