Law of Large Numbers (LLN): A Visualization

Flipping a coin 1,000 times, you might know to expect about 500 heads (since we have a 50/50 chance for heads/tails). This isn't the only outcome though. We could flip 1,000 heads and be considered unlucky indeed (the chance this happens is about 1 over a googol cubed). What the law of large numbers says is that these unfortunate cases probably won't affect the average in the long run.

Try generating some random numbers uniformly yourself or clicking 'Start Random' to automatically generate uniformly. What we expect is that the average will eventually converge to the mean.

Number of options:

Previous sample: - Sample count: 0 Average: 0

Formally, what the Weak Law says is that the running average is arbitrarily close to the mean with probability 1.

Weak LLN

Let $X_1,X_2,\dots$ be independent and identically distributed random variables with a finite expected value ($E[X]<\infty$). Then, for any $\epsilon>0$, $$\lim_{n\to \infty}\mathbb{P}\left[\left|\frac{1}{n}\sum_{i=1}^nX_i-\mu\right|\geq \epsilon \right]=0$$

It's more profound counterpart, the Strong Law, states the limit of the average is the mean with probability 1.

Strong LLN

Let $X_1,X_2,\dots$ be independent and identically distributed random variables with a finite mean and variance. Then, $$\mathbb{P}\left[\lim_{n\to \infty}\frac{1}{n}\sum_{i=1}^nX_i=\mu \right]=1$$

Garrett Kepler