Comparison Of Designs 28

1. Core Concept Overview & Intuition

The comparison of designs is the branch of experimental design theory that answers a practical question: given several candidate field plans (completely randomized, randomized block, Latin square, balanced incomplete block, and so on), which one should be adopted for a given experiment? The criterion for selection is not a single number but a structured judgement about how much of the total variability in the response the design is able to remove from the error term, and at what cost in degrees of freedom, plots, and assumptions.

The organizing idea is the partition of total variation. Every design studied so far decomposes the total sum of squares into a component attributable to treatments and one or more components attributable to nuisance structure (blocks, rows, columns), with a residual error component. A design is preferred when it assigns a large share of the nuisance variability to identifiable sources, leaving a smaller error mean square and hence a more powerful test of treatment equality. The completely randomized design removes nothing beyond the treatment effect; the randomized block design removes block-to-block variation; the Latin square design removes variation in two directions simultaneously. Each successive refinement buys precision at the price of stronger assumptions and tighter restrictions on the number of treatments and replicates.

The comparison is therefore conducted through the analysis of variance of each design, using the expected mean squares and the resulting $F$-ratios. Because the $F$-ratio for treatments is the ratio of the treatment mean square to the error mean square, any design that reduces the error mean square while preserving the treatment contrast increases the power of the test. The comparison of designs is thus, in essence, a comparison of error mean squares and of the degrees of freedom attached to them.

A second, equally important dimension of the criterion is the validity of the design under the field conditions actually available. A design that is theoretically more efficient but requires homogeneity assumptions that the field cannot satisfy is inferior in practice. The criterion for selection therefore balances statistical efficiency, the number of error degrees of freedom, the feasibility of randomization, and the robustness of the design to departures from its assumptions.

2. Mathematical Formalization & Core Definitions

> Definition (Design comparison). Let two or more candidate designs be available for an experiment on $v$ treatments. The comparison of designs is the systematic evaluation of their relative precision for estimating and testing treatment effects, carried out through the analysis of variance of each design and the comparison of the error mean squares and treatment $F$-ratios so obtained.

> Definition (Criterion for selection). The criterion for selection is the rule by which one design is declared preferable to another. It is applied by comparing the error mean squares of the competing designs, the degrees of freedom associated with error, and the assumptions each design imposes on the experimental material.

The formal machinery rests on the linear additive model and its least-squares analysis, already established for the completely randomized design. For the completely randomized design with $v$ treatments and $n_i$ observations on the $i$th treatment, the model is

$$
y_{ij}=\mu+t_i+e_{ij},\qquad e_{ij}\sim N(0,\sigma^2),
$$

where $\mu$ is the general mean, $t_i$ is the effect of the $i$th treatment, $y_{ij}$ is the $j$th observation on the $i$th treatment, and $e_{ij}$ is the error variable. The least-squares estimates are

$$
\hat{\mu}=\bar{y}{.},\qquad \hat{t}_i=\bar{y},}-\bar{y}_{.
$$

with $\bar{y}{.}$ the grand mean and $\bar{y}$ the mean of the $i$th treatment. The partition of the total sum of squares is

$$
\sum_{i=1}^{v}\sum_{j=1}^{n_i}(y_{ij}-\bar{y}{.})^2=\sum}^{v}n_i(\bar{y{i.}-\bar{y})^2,})^2+\sum_{i=1}^{v}\sum_{j=1}^{n_i}(y_{ij}-\bar{y}_{i.
$$

written compactly as

$$
TSS=SSTMT+SSE.
$$

The analysis of variance table for the completely randomized design is the reference against which the other designs are judged.

Here $n=\sum_{i=1}^{v}n_i$ is the total number of plots, $T$ denotes the treatment mean square, and $E$ denotes the error mean square. The test statistic is

$$
F_c=\frac{T}{E}=\frac{MSTMT}{MSE},
$$

and $H_0:t_1=t_2=\cdots=t_v=0$ is rejected at level $\alpha$ whenever $F_c>F_\alpha(n_1,n_2)$, the tabular value with $n_1=v-1$ and $n_2=n-v$ degrees of freedom.

The computational formulas used in every comparison are

$$
TSS=\sum_{i=1}^{v}\sum_{j=1}^{n_i}y_{ij}^2-\frac{G^2}{n},
$$

where

$$
G=\sum_{i=1}^{v}\sum_{j=1}^{n_i}y_{ij}
$$

is the grand total and $G^2/n$ is the correction factor, and

$$
SSTMT=\sum_{i=1}^{v}\frac{T_i^2}{n_i}-\frac{G^2}{n},\qquad T_i=\sum_{j=1}^{n_i}y_{ij}=n_i\bar{y}_{i.},
$$

with

$$
SSE=TSS-SSTMT.
$$

These quantities are computed identically for each candidate design; what changes from design to design is the number and nature of the nuisance components that are subtracted before the error term is formed.

> Definition (Efficiency of a design). The efficiency of one design relative to another is the ratio of the error mean square of the less structured design to the error mean square of the more structured design, both computed on the same experimental material. A ratio greater than one indicates that the more structured design has removed nuisance variation and is therefore the more precise.

The expected mean squares supply the theoretical basis for the comparison. For the completely randomized design, the error sum of squares satisfies

$$
E\left[\sum_{i=1}^{v}\sum_{j=1}^{n_i}(y_{ij}-\bar{y}_{i.})^2\right]=(n-v)\sigma^2,
$$

so that

$$
E\left{\frac{W}{n-v}\right}=\sigma^2,
$$

where

$$
W=\frac{1}{\sigma^2}\sum_{i=1}^{v}\sum_{j=1}^{n_i}(y_{ij}-\bar{y}_{i.})^2.
$$

The treatment sum of squares has expectation

$$
E\left[\sum_{i=1}^{v}n_i(\bar{y}{i.}-\bar{y})^2,})^2\right]=(v-1)\sigma^2+\sum_{i=1}^{v}n_i(t_i-\bar{t
$$

which is unbiased for $(v-1)\sigma^2$ precisely when $H_0:t_1=\cdots=t_v$ holds. The ratio of the two mean squares therefore has expectation near one under the null hypothesis and exceeds one under the alternative, which is the mechanism that makes the $F$-test a valid instrument of comparison.