In statistics, Cochran's theorem, devised by William G. Cochran, is a theorem used to justify results relating to the probability distributions of statistics that are used in the analysis of variance.
Let be i.i.d. standard normally distributed random variables, and . Let be symmetric matrices. Define r<sub>i</sub> to be the rank of . Define , so that the Q<sub>i</sub> are quadratic forms. Further assume .
Cochran's theorem states that the following are equivalent:
Often it's stated as , where is idempotent, and is replaced by . But after an orthogonal transform, , and so we reduce to the above theorem. Here, denotes the diagonal transform.
The following version is often seen when considering linear regression. Suppose that is a standard multivariate normal random vector (here denotes the n-by-n identity matrix), and if are all n-by-n symmetric matrices with . Then, on defining , any one of the following conditions implies the other two:
If X<sub>1</sub>, ..., X<sub>n</sub> are independent normally distributed random variables with mean ü and standard deviation àthen
is standard normal for each i. Note that the total Q is equal to sum of squared Us as shown here:
which stems from the original assumption that . So instead we will calculate this quantity and later separate it into Q<sub>i</sub>'s. It is possible to write
(here is the sample mean). To see this identity, multiply throughout by and note that
and expand to give
The third term is zero because it is equal to a constant times
and the second term has just n identical terms added together. Thus
and hence
Now with the matrix of ones which has rank 1. In turn given that . This expression can be also obtained by expanding in matrix notation. It can be shown that the rank of is as the addition of all its rows is equal to zero. Thus the conditions for Cochran's theorem are met.
Cochran's theorem then states that Q<sub>1</sub> and Q<sub>2</sub> are independent, with chi-squared distributions with n − 1 and 1 degree of freedom respectively. This shows that the sample mean and sample variance are independent. This can also be shown by Basu's theorem, and in fact this property characterizes the normal distribution â for no other distribution are the sample mean and sample variance independent.
The result for the distributions is written symbolically as
Both these random variables are proportional to the true but unknown variance ÃÂ<sup>2</sup>. Thus their ratio does not depend on ÃÂ<sup>2</sup> and, because they are statistically independent. The distribution of their ratio is given by
where F<sub>1,n − 1</sub> is the F-distribution with 1 and n − 1 degrees of freedom (see also Student's t-distribution). The final step here is effectively the definition of a random variable having the F-distribution.
To estimate the variance ÃÂ<sup>2</sup>, one estimator that is sometimes used is the maximum likelihood estimator of the variance of a normal distribution
Cochran's theorem shows that
and the properties of the chi-squared distribution show that
Claim: Let be a standard Gaussian in , then for any symmetric matrices , if and have the same distribution, then have the same eigenvalues (up to multiplicity).
Claim: .
Lemma: If , all symmetric, and have eigenvalues 0, 1, then they are simultaneously diagonalizable.
Now we prove the original theorem. We prove that the three cases are equivalent by proving that each case implies the next one in a cycle ().