Special Discrete Probability Distributions

1. Core Concept Overview & Intuition

The three distributions treated here are the fundamental models for counting the number of "successes" in a finite sequence of trials. They share a common logical structure but differ in one decisive respect: whether trials are independent and whether the probability of success stays constant from trial to trial.

The Bernoulli distribution is the atom of the family. A single trial is performed, and the outcome is classified into exactly two mutually exclusive categories, conventionally called success and failure. The random variable records success as $1$ and failure as $0$. Every richer model below is built by aggregating independent or dependent Bernoulli-type trials.

The binomial distribution arises when a fixed number $n$ of independent Bernoulli trials, each with the same success probability $p$, are performed and the random variable counts the total number of successes. Independence and a constant $p$ are the defining assumptions. Typical settings include repeated independent selection with replacement, or sampling from a population so large that the removal of a few items does not materially change the composition.

The hypergeometric distribution arises when sampling is performed without replacement from a finite population of known composition. Here the trials are not independent: each draw alters the proportion of successes remaining in the population. The random variable counts the number of successes (for instance, defective items or women on a committee) in a sample of fixed size drawn from a finite population containing a known number of successes.

The practical distinction is therefore one of sampling scheme. Sampling with replacement, or sampling from an effectively infinite population, produces the binomial model. Sampling without replacement from a small finite population produces the hypergeometric model. The two models converge in behaviour as the population size grows large relative to the sample size, because removing a small fraction of a large population barely changes its composition.

For each distribution the central analytical tasks are the same: verify that the proposed function is a legitimate probability distribution, derive its mean and variance, and obtain its moment generating function, from which the moments may be recovered by differentiation at zero.

2. Mathematical Formalization & Core Definitions

2.1 Preliminaries: Conditions for a Discrete Probability Distribution

> Definition (Discrete probability distribution). A real-valued function $f(x)$ is the probability distribution of a discrete random variable $X$ if it satisfies two conditions:

> (i) $f(x)\ge 0$ for every value $x$ in the support of $X$;

> (ii) $\sum_{x} f(x)=1$, the sum being taken over all values $x$ for which $f(x)>0$.

These two conditions are the standard test applied whenever a candidate function, possibly containing an unknown constant, is proposed as a distribution. The constant is fixed by imposing condition (ii).

Throughout, $q$ denotes the failure probability, defined by $q=1-p$, so that $p+q=1$.

2.2 The Bernoulli Distribution

> Definition (Bernoulli distribution). A random variable $X$ has a Bernoulli distribution if its discrete density function is

$$
f(x)=\begin{cases} p^{x}(1-p)^{1-x}, & x=0,1,\ q=1-p \ 0, & \text{otherwise} \end{cases}
$$

The parameter is $p$, with $0\le p\le 1$. The support consists of the two points $0$ and $1$ only. At $x=1$ the density equals $p$; at $x=0$ it equals $1-p=q$.

2.3 The Binomial Distribution

> Definition (Binomial distribution). A random variable $X$ has a binomial distribution if its discrete density is

$$
f(x)=\begin{cases} \binom{n}{x}p^{x}(1-p)^{n-x}, & x=0,1,\dots,n,\ q=1-p \ 0, & \text{otherwise} \end{cases}
$$

The parameters are $n$ and $p$, where $n$ is a positive integer and $0\le p\le 1$. The support is the finite set ${0,1,\dots,n}$. The binomial distribution is also referred to as the Bernoulli distribution in the broad sense, since it is the distribution of a sum of $n$ independent Bernoulli trials.

2.4 The Hypergeometric Distribution

> Definition (Hypergeometric distribution). Suppose a box contains $N$ items, of which $M$ are defective, and $n$ items are selected at random without replacement. Let $X$ be the number of defective items in the sample. Then $X$ is a hypergeometric random variable with probability distribution

$$
f(x)=P(X=x)=\begin{cases} \dfrac{\binom{M}{x}\binom{N-M}{n-x}}{\binom{N}{n}}, & x=0,1,2,\dots,n \ 0, & \text{otherwise} \end{cases}
$$

Here $N$ is a positive integer, $M$ is a non-negative integer that is at most $N$, and $n$ is a positive integer that is at most $N$. The numerator counts the ways of choosing $x$ defective items from the $M$ available and $n-x$ non-defective items from the $N-M$ available; the denominator counts all ways of choosing the sample of size $n$ from the $N$ items. The support is bounded below by $\max(0,\,n-(N-M))$ and above by $\min(n,\,M)$, since a sample cannot contain more defectives than exist or more non-defectives than exist.

2.5 Moment Generating Functions

> Definition (Moment generating function). For a discrete random variable $X$, the moment generating function is $M_X(t)=E(e^{tX})=\sum_{x} e^{tx}f(x)$, defined for all real $t$ for which the sum converges.

The moment generating functions of the three distributions are:

$$
M_X(t)=(1-p)+pe^{t}\quad\text{(Bernoulli)}
$$

$$
M_X(t)=(pe^{t}+q)^{n}\quad\text{(binomial)}
$$

The Bernoulli moment generating function exists for all real numbers $t$. The binomial moment generating function follows from the binomial theorem applied to the sum $\sum_{x=0}^{n}\binom{n}{x}(pe^{t})^{x}(1-p)^{n-x}$.