In a discussion on emailmarketersclub.com I just stumbled over some interesting benchmarks that have been published by MailChimp a while ago. Email senders have always been keen on comparing themselves to others in order know where they stand. The reported data, although from 2010, might be a good reference for small enterprises. From a consultant perspective, it would also be interesting to know if there are industry groups that perform similarly good or bad. Well, have a look at my fancy little email marketing benchmark cube…
Industry benchmarks in 3D:
… Usage: Drag the mouse to rotate the cube, use your mouse wheel to zoom in and out, and the right mouse button allows you to change the field of view.)
- The width of the cube, i.e. the x-axis entitled “[pc2] OPENS&CLICKS“, represents a combined measure for open and click rates: A value of three is superb, minus one is bad. The y-axis (height) is a combined measure for hard and soft bounces. The z-axis (depth) called “[pc3] UNSUBSCRIBES” represents the unsubscribe rates for the different industries.
- Something to the scale: all values are mean-centered, i.e. 0 represents the average performance for the corresponding measure. Going to -1 is under average and to +1 over average correspondingly. Take for example “Other”, which ranks highest for the open and click rate measure: it also has average bounce rates (pc1 close to 0) and unsubscribe rates slightly above the average (pc3 close to 1).
- You might have noticed the half transparent plane. This marks the mean average for the combined measure of open and click rates. Therefore, industries on the left of it underperform, whereas the intersecting ones have rather average open and click rates. Finally, those few industries on the right of the plane rank above the global average.
- The colors represent the five clusters that I found (plus two outliers “Other” and “Social Network and Online Communities”). Clusters should have rather homogenous pc1, pc2 & pc3 values within themselves, and rather heterogeneous values in comparison to one another.
(A little YouTube-experiment. It may be hard to listen to because of my rusty english, but I gave it a try anyway… )
Other cluster solutions:
Without diving deeper into cluster quality measures, other possible solutions for visual inspection may include for instance two or seven segments:
How was it done?
In case you are interested in how the cube was constructed:
- First, a correlation analysis was done, which showed significant relationships between several measures The results in form of o correlogram:
- A principal component analysis (PCA) was run to reduce the number of dimensions. The original data has six dimensions – ranging from open rates to unsubscribe rates. “Abuse complaint rate” was excluded here, because it had only little in common with the others. Three components pc1 to pc3 were extracted (for 3D modeling) and rotated. All three together account for 91 percent of variance. The resulting biplot matrix including factor loadings:
- Then, a cluster analysis was run. First, single linkage clustering identified possible outliers. Then, ward clustering was performed with the remaining industries. The resulting dendrogram is shown above.
- The cube was rendered and exported using the wonderful rgl package in R.
Those steps resulted in three principal components (including their factor scores), a cluster attribute, and the 3D cube. Note that I didn’t bother about validating the clustering and PCA results any further. Also, the requirements for a PCA weren’t perfectly met (e.g. there were only 37 cases, outliers…)
Nevertheless, the procedure itself could of course also be translated to, let’s say, inspecting last year’s email campaigns. Maybe there are clusters of similarly successful campaigns? The answers to many questions like this lie within your data. It just got to be used – in the right way.