Statistical Significance

Confidence that a measured difference is real rather than noise. Requires more data than small-site tests usually have.

Free · no signup · last reviewed August 29, 2026

The definition

Confidence that a measured difference is real rather than noise. Requires more data than small-site tests usually have.

Why it matters to an owner

The protection against fooling yourself with a decimal point: under ~100 conversions per variant, your A/B test is astrology. Make bolder changes and judge over longer windows instead.

This is the work. Want it done?

Everything here is free to use yourself — that’s the point. If you’d rather a senior marketer just handle it, in your accounts and your name: 20-minute call, no pitch deck.

Contact