Characteristic Functions

1. Core Concept Overview & Intuition

The characteristic function is a transform of a probability distribution that converts the operation of convolution of distributions into ordinary multiplication of functions, and it converts weak convergence of distributions into pointwise convergence of functions. For a random variable $X$ with distribution function $F(x)$, the characteristic function is the expectation of the complex exponential $e^{itX}$, where $t$ is a real parameter and $i$ is the imaginary unit. Because the complex exponential decomposes into a cosine part and a sine part, the characteristic function is a complex-valued function whose real and imaginary components are expectations of bounded continuous functions. This boundedness is the source of the two most useful structural facts: the characteristic function always exists (the integral converges absolutely), and it is uniformly continuous in $t$.

The intuition behind the transform is that the distribution function $F$ encodes the probability of every interval, while the characteristic function encodes the same information in a form adapted to independence and to limiting arguments. When independent random variables are added, their characteristic functions multiply; when a distribution is shifted or rescaled, the characteristic function is modulated by a complex exponential and rescaled in its argument. The inversion theorem makes the encoding reversible: integrating the characteristic function against a suitable kernel over the whole real line recovers the probability of any interval whose endpoints are continuity points of $F$. The remaining material in this unit supplies the probabilistic scaffolding needed to state and use these facts: the algebra of events, conditional probability, independence of events and of classes of events, independence of random variables through the $\sigma$-fields they induce, and the tail $\sigma$-field that records events depending only on the remote future of a sequence.

A useful mental picture is that of a distribution being probed by a family of oscillating test functions $x \mapsto e^{itx}$ indexed by frequency $t$. Low frequencies see the gross location of the mass; high frequencies see fine structure. The continuity of $Q_X$ in $t$ says that small changes in frequency produce small changes in the probe, uniformly over all distributions, because the probe is bounded by one in modulus. The bound $\lvert Q(t)\rvert \le 1$ says that no distribution can produce an average oscillation exceeding unit magnitude, since each individual oscillation has modulus exactly one.

2. Mathematical Formalization & Core Definitions

2.1 The characteristic function

> Definition (Characteristic function). Let $X$ be a random variable with distribution function $F(x)$. The characteristic function of $X$ is

$$
Q_X(t) = E\left(e^{itX}\right) = \int e^{itx}\,dF(x),
$$

> where $t \in \mathbb{R}$ and the integral is a Lebesgue–Stieltjes integral with respect to $F$.

Because $e^{itx} = \cos(tx) + i\sin(tx)$, the characteristic function splits into real and imaginary parts:

$$
Q_X(t) = \int \cos(tx)\,dF(x) + i\int \sin(tx)\,dF(x).
$$

The real part is the expectation of $\cos(tX)$ and the imaginary part is the expectation of $\sin(tX)$. Both integrands are bounded by one in absolute value, so both integrals exist for every distribution $F$ and every $t$.

2.2 Foundational set-theoretic and probabilistic definitions

The transform is developed on a probability space whose events form a $\sigma$-field. The following definitions supply the vocabulary used throughout.

> Definition (Field and $\sigma$-field). A class $\mathcal{F}$ of subsets of a sample space $U$ is a field if it contains $U$, is closed under complementation, and is closed under finite unions. It is a $\sigma$-field if it is additionally closed under countable unions.

> Definition (DeMorgan's rule). For any class of sets ${A_\alpha}$,

$$
\left(\bigcup_\alpha A_\alpha\right)^c = \bigcap_\alpha A_\alpha^c, \qquad \left(\bigcap_\alpha A_\alpha\right)^c = \bigcup_\alpha A_\alpha^c.
$$

DeMorgan's rule is the mechanism by which closure under unions becomes closure under intersections, and it is used repeatedly when passing between an event and its complement, for example in the second Borel–Cantelli argument where $\bigcap_k B_k^c$ is the complement of $\bigcup_k B_k$.

> Definition (Function and random variable). A function $f$ from a domain $U$ to a codomain $V$ assigns to each $u \in U$ exactly one value $f(u) \in V$. A random variable $X$ is a function on $U$ such that for every Borel set $B$ the preimage $X^{-1}B$ is an event.

> Definition (Probability). A probability $P$ is a set function on a $\sigma$-field $\mathcal{F}$ with $P(U) = 1$, $P(A) \ge 0$ for all $A \in \mathcal{F}$, and countable additivity: for pairwise disjoint events ${A_n}$,

$$
P\left(\bigcup_n A_n\right) = \sum_n P(A_n).
$$

> Definition (Approaches to probability). The classical approach assigns probabilities to equally likely outcomes by counting; the relative-frequency approach defines probability as the limiting proportion of occurrences in repeated trials; the axiomatic approach takes the three axioms above as primitive and derives all further properties.

> Definition (Expectation). For a random variable $X$ with distribution function $F$, the expectation is

$$
E(X) = \int x\,dF(x),
$$

> whenever the integral exists, and for a measurable function $h$ the expectation of $h(X)$ is $E(h(X)) = \int h(x)\,dF(x)$.

The characteristic function is precisely the expectation of the complex-valued function $h(x) = e^{itx}$ evaluated at $X$, so its definition is a direct instance of the expectation definition.

> Definition (Sequence and limit). A sequence ${x_n}$ converges to $x$ if for every $\epsilon > 0$ there exists $N$ such that $\lvert x_n - x\rvert < \epsilon$ for all $n \ge N$. A sequence of random variables ${X_n}$ converges in probability to $X$ if for every $\epsilon > 0$,

$$
P\left(\lvert X_n - X\rvert > \epsilon\right) \to 0.
$$

2.3 Conditional probability and independence of events

> Definition (Conditional probability). For events $A$ and $B$ with $P(A) > 0$, the conditional probability of $B$ given $A$ is

$$
P(B \mid A) = \frac{P(AB)}{P(A)},
$$

> and consequently $P(AB) = P(A)P(B \mid A)$.

> Definition (Independence of two events). Events $A$ and $B$ are independent if $P(B \mid A) = P(B)$, equivalently if

$$
P(AB) = P(A)P(B).
$$

Three consequences are recorded in the source. First, if $A$ and $B$ are independent then $A$ and $B^c$ are also independent. Second, mutually exclusive events with $P(A) \ne 0$ and $P(B) \ne 0$ are always dependent, because $P(AB) = 0$ while $P(A)P(B) \ne 0$. Third, the null event $N$ with $P(N) = 0$ is independent of every event $A$, since

$$
P(AN) = P(A)P(N \mid A) = P(A)\cdot 0 = 0 = P(A)P(N),
$$

and $N$ is independent of itself since $P(NN) = 0\cdot 0 = P(N)P(N)$. Likewise the sure event $N^c$ with $P(N^c) = 1$ is independent of every event $A$, since

$$
P(AN^c) = P(A)P(N^c \mid A) = P(A)\cdot 1 = P(A)P(N^c),
$$

and $N^c$ is independent of itself.

2.4 Classes of independent events and independent classes

> Definition (Mutually independent class of events). Let $\mathcal{C} = {A_1, A_2, \dots, A_n}$ be a class of events and let ${A_{j_1}, A_{j_2}, \dots, A_{j_k}}$ be any sub-class. The class $\mathcal{C}$ is mutually independent if for every $k \le n$,

$$
P(A_{j_1}A_{j_2}\cdots A_{j_k}) = \prod_i P(A_{j_i}).
$$

> Definition (Independent classes). Two classes $\mathcal{C}_1$ and $\mathcal{C}_2$ are independent if for every $A_i \in \mathcal{C}_1$ and $B_j \in \mathcal{C}_2$,

$$
P(A_iB_j) = P(A_i)P(B_j).
$$

> Classes $\mathcal{C}1, \mathcal{C}_2, \dots, \mathcal{C}_q$ are independent if for every choice $A} \in \mathcal{C1, \dots, A_q$,} \in \mathcal{C

$$
P(A_{i_1}A_{i_2}\cdots A_{i_q}) = \prod_j P(A_{i_j}).
$$

2.5 Independent random variables and induced $\sigma$-fields

> Definition (Mutually independent random variables). A set ${X_t: t = 1, 2, \dots, T}$ of random variables is mutually independent if the $\sigma$-fields induced by the random variables are mutually independent.

> Definition ($\sigma$-field induced by a finite sequence). For random variables $X_1, X_2, \dots, X_n$, the induced $\sigma$-field is the minimal $\sigma$-field containing all the $\sigma$-fields induced by each of them:

$$
\mathcal{A}_n = \sigma\left[\bigcup_i X_i^{-1}\mathcal{B}\right],
$$

> where $\mathcal{B}$ denotes the Borel $\sigma$-field on the real line.

> Definition (Tail $\sigma$-field). Let $D_n$ be the $\sigma$-field induced by $(X_n, X_{n+1}, \dots)$, so that

$$
D_n = \sigma\left[\bigcup_{i \ge n} X_i^{-1}\mathcal{B}\right].
$$

> Since $D_n$ contains $D_{n+1}$, the sequence ${D_n}$ is monotone decreasing, and its limit is

$$
\tau = \bigcap_{n=1}^{\infty} D_n.
$$

> The class $\tau$ is a $\sigma$-field because it is an intersection of $\sigma$-fields; it is called the tail $\sigma$-field and its events are called tail events. Every $D_n$ contains $\tau$.

2.6 The lemma on the real part of the characteristic function

> Lemma. For any characteristic function $Q$,

$$
\operatorname{Re}\left(1 - Q(t)\right) \ge \frac{1}{4}\operatorname{Re}\left(1 - Q(2t)\right),
$$

> where $\operatorname{Re}Q(t)$ denotes the real part of $Q(t)$.

The proof uses the reality of the distribution function. Starting from the double-angle identity for cosine,

$$
1 - \operatorname{Re}Q(2t) = \int \left(1 - \cos(2tx)\right)dF(x) = 2\int\left(1 - \cos^2(tx)\right)dF(x),
$$

and factoring the difference of squares,

$$
1 - \operatorname{Re}Q(2t) = 2\int\left(1 - \cos(tx)\right)\left(1 + \cos(tx)\right)dF(x).
$$

Since $1 + \cos(tx) \le 2$,

$$
1 - \operatorname{Re}Q(2t) \le 4\int\left(1 - \cos(tx)\right)dF(x) = 4\operatorname{Re}\left(1 - Q(t)\right),
$$

which rearranges to the stated inequality.