Tag Archives: Test

When testing, it’s a good idea to have some formulas to hand. For instance in split A/B/n test scenarios, you may want to inspect the relationship between sample size, level of significance, and power. Also when renting lists, no one likes to buy a pig in a poke. Instead, the campaign has to be tested on a small segment first. Only if the test turns out to provide a good return on investment, the full run will be booked.

However, the question is, how many recipients should one book for the test? Including too many recipients would only cost in case the list proves to be unprofitable. Renting too few subscribers on the other hand bears the risk that the test results are due to chance. Here’s a hands-on solution. Continue reading →

In one of the last posts, we addressed the chi-squared test for independence. With this test, we wanted to calculate, if e.g. two subject lines have a significantly different impact on the absolute number of email opens. I provided you with a “flexible” solution. “Flexible” means, it can now easily be extended to your needs. One extension would be to determine the required sample size for each of your test cells a priori. There’s no question that a split A/B test, which only incorporates 2 x 100 recipients delivers a different reliability than one, which includes 2 x 1500 recipients. So here’s a solution for choosing the right sample size — again using the R package.
Continue reading →