my-server
← Wiki

Law of total variance

The law of total variance is a fundamental result in probability theory that expresses the variance of a random variable in terms of its conditional variances and conditional means given another random variable . Informally, it states that the overall variability of can be split into an “unexplained” component (the average of within-group variances) and an “explained” component (the variance of group means).

Formally, if and are random variables on the same probability space, and has finite variance, then:

This identity is also known as the variance decomposition formula, the conditional variance formula, the law of iterated variances, or colloquially as Eve’s law, in parallel to the “Adam’s law” naming for the law of total expectation.

In actuarial science (particularly in credibility theory), the two terms and are called the expected value of the process variance (EVPV) and the variance of the hypothetical means (VHM) respectively.

Explanation

Let be a random variable and another random variable on the same probability space. The law of total variance can be understood by noting:

  1. measures how much varies around its conditional mean
  2. Taking the expectation of this conditional variance across all values of gives , often termed the “unexplained” or within-group part.
  3. The variance of the conditional mean, , measures how much these conditional means differ (i.e. the “explained” or between-group part).

Adding these components yields the total variance , mirroring how analysis of variance partitions variation.

Examples

Example 1 (Exam scores)

Suppose five students take an exam scored 0–100. Let = student’s score and indicate whether the student is *international* or *domestic*:

  • Mean and variance for international:
  • Mean and variance for domestic:

Both groups share the same mean (50), so the explained variance is 0, and the total variance equals the average of the within-group variances (weighted by group size), i.e. 800.

Example 2 (Mixture of two Gaussians)

Let be a coin flip taking values with probability and with probability . Given Heads, ; given Tails, . Then

so

Example 3 (Dice and coins)

Consider a two-stage experiment:

  1. Roll a fair die (values 1–6) to choose one of six biased coins.
  2. Flip that chosen coin; let =1 if Heads, 0 if Tails.

Then The overall variance of becomes

with uniform on

Proof

Discrete/finite proof

Let , , be observed pairs. Define Then

where Expanding the square and noting the cross term cancels in summation yields:

General case

Using and the law of total expectation:

Subtract and regroup to arrive at

Applications

Analysis of Variance (ANOVA)

In a one-way analysis of variance, the total sum of squares (proportional to ) is split into a “between-group” sum of squares () plus a “within-group” sum of squares (). The F-test examines whether the explained component is sufficiently large to indicate has a significant effect on .

Regression and R²

In linear regression and related models, if the fraction of variance explained is

In the simple linear case (one predictor), also equals the square of the Pearson correlation coefficient between and .

Machine learning and Bayesian inference

In many Bayesian and ensemble methods, one decomposes prediction uncertainty via the law of total variance. For a Bayesian neural network with random parameters :

often referred to as “aleatoric” (within-model) vs. “epistemic” (between-model) uncertainty.

Actuarial science

Credibility theory uses the same partitioning: the expected value of process variance (EVPV), and the variance of hypothetical means (VHM), The ratio of explained to total variance determines how much “credibility” to give to individual risk classifications.

Information theory

For jointly Gaussian , the fraction relates directly to the mutual information In non-Gaussian settings, a high explained-variance ratio still indicates significant information about contained in .

Generalizations

The law of total variance generalizes to multiple or nested conditionings. For example, with two conditioning variables and :

More generally, the law of total cumulance extends this approach to higher moments.

See also

References