Imagine that before your next newsletter goes out, you want to know which one of your two call to action ideas attracts more clicks: “Buy now!” (a) or “See more!” (b). Therefore, you conduct a split a/b pretest. One hour later, the results are in. Within your control group (a), 100 of 1.000 recipients clicked a link. The test group (b) reveals a 15% unique click rate – i.e. even 150 recipients clicked…
Question: Can call to action (b) really attribute for +50 clickers, or was the difference in click rates between (a) and (b) due to chance?
And what, if we compared a 10% to a 11% unique click rate? Or if we’d check 200 versus 220 unique opens? Or 10 vs. 20 unsubscribes?
Validate your tests:
The split test calculator below performs Pearson’s chi-squared test (not Yates’). It answers such questions easily. Just enter your test results and the number of recipients for your two splits. Specify your results either as absolute unique values or as percentages. For example, “
100” could be 100 unique clicks, “
15.0” could be 15% unique click rate.