Statistical Significance

Confidence that a measured difference is real rather than noise. Requires more data than small-site tests usually have.

Free · no signup · last reviewed August 29, 2026

The definition

Confidence that a measured difference is real rather than noise. Requires more data than small-site tests usually have.

Why it matters to an owner

The protection against fooling yourself with a decimal point: under ~100 conversions per variant, your A/B test is astrology. Make bolder changes and judge over longer windows instead.

This is the work. Want it done?

Everything here is free to use yourself. That’s the point. If you’d rather a senior marketer just handle it, in your accounts and your name: 20-minute call, no pitch deck.

Contact