Tag Archives: Test

How animated symbols in Gmail subject lines increased my open rate

As announced in the “Email Gurus” board on LinkedIn, I put Gmail’s animated emoji symbols to the subject line test last Friday:

Here’s how I did it and what the results were. So much is revealed, they’re like…
Continue reading

Simplest scientific split test calculator for email senders [Tool]

splittestcalcImagine that before your next newsletter goes out, you want to know which one of your two call to action ideas attracts more clicks: “Buy now!” (a) or “See more!” (b). Therefore, you conduct a split a/b pretest. One hour later, the results are in. Within your control group (a), 100 of 1.000 recipients clicked a link. The test group (b) reveals a 15% unique click rate – i.e. even 150 recipients clicked…

Question: Can call to action (b) really attribute for +50 clickers, or was the difference in click rates between (a) and (b) due to chance?
And what, if we compared a 10% to a 11% unique click rate? Or if we’d check 200 versus 220 unique opens? Or 10 vs. 20 unsubscribes? Continue reading

Multivariate tests in email marketing – a practical step-by-step guide to boost your response rates

Multivariate testing is a bit underrated. Marketing weblogs mostly focus on A/B or A/B/n tests. Those are quickly set up. But they often provide only incremental gains. MVT are more promising with regards to the outcome. Let’s look at how they work. Continue reading

[Split-Test] Email signups doubled AND bounces reduced at the same time – see how

I want to share some testing results with you. For a couple of weeks, I ran a split A/B/C/D test on this website. The goal was to see, if I could convert more visitors to email subscribers. “Which test won”… can you guess it? The results are quite interesting. Continue reading

Email pre-testing: Determining required group sizes and margins of error

When testing, it’s a good idea to have some formulas to hand. For instance in split A/B/n test scenarios, you may want to inspect the relationship between sample size, level of significance, and power. Also when renting lists, no one likes to buy a pig in a poke. Instead, the campaign has to be tested on a small segment first. Only if the test turns out to provide a good return on investment, the full run will be booked.

However, the question is, how many recipients should one book for the test? Including too many recipients would only cost in case the list proves to be unprofitable. Renting too few subscribers on the other hand bears the risk that the test results are due to chance. Here’s a hands-on solution. Continue reading

Determining statistical significance for email split tests, pt. 2: sample sizes

In one of the last posts, we addressed the chi-squared test for independence. With this test, we wanted to calculate, if e.g. two subject lines have a significantly different impact on the absolute number of email opens. I provided you with a “flexible” solution. “Flexible” means, it can now easily be extended to your needs. One extension would be to determine the required sample size for each of your test cells a priori. There’s no question that a split A/B test, which only incorporates 2 x 100 recipients delivers a different reliability than one, which includes 2 x 1500 recipients. So here’s a solution for choosing the right sample size — again using the R package.
Continue reading

GetElastic: 5 Tipps fürs Testen von E-Mails

Linda Bustos vom kanadischen E-Commerce-Dienstleister Elastic Path verrät in ihrem Get Elastic-Blog in einem 10-minütigen Präsentationsvideo einige Tipps und Tricks plus nützliche Links zum sinnvollen Testen von werblichen E-Mails:

[vodpod id=Groupvideo.2298741&w=425&h=350&fv=]


  1. Haupt-Unterschiede fokussieren (Makro- vs. Mikrovariablen)
  2. 5:00 min: Segmentieren
  3. 7:15 min: Passende Landingpages nutzen
  4. 8:00 min: Offline-Conversions verfolgen
  5. 8:35 min: Testen, testen, testen…

(Via BeRelevant!)

Test: CSS-Support von BlackBerry, iPhone und Treo

Test-Ergebnisse für Blackberries & iPhones

Abb.: Support gängiger CSS-Attribute (Quelle: G. Oldring)

Die stärkere Berücksichtigung mobiler Endgeräte im Rahmen des E-Mail-Marketings sehen viele Marketer als große Herausforderung für das angelaufene Jahr 2009.1 Zahlreiche Statistiken belegen zudem die stetig wachsende Bedeutung der mobilen Endgeräte für die E-Mail-Korrespondenz.2 3

Greg Oldring hat sich die Mühe gemacht und mit einem Acid-Test [?] einige mobile Clients in seinem Blog unter die Lupe genommen. Aus den Ergebnissen (vgl. Abb.) ergeben sich ganz konkrete Empfehlungen für das E-Mail-Design für Smartphones.

Die getesteten Modelle: Continue reading

Whitepaper: Tipps zum Testen im E-Mail-Marketing

Im aktuellen Interview “E-Mail-Marketing Rückblick und Ausblick” von Nico Zorn mit Uwe-Michael Sinn (Geschäftsführer von rabbit eMarketing) betont Sinn die Wichtigkeit einer

systematische(n) Durchführung von Testkampagnen. Welches Design Continue reading