Enhance your emails with a self-coded live twitter feed

How about some realtime content in your emails? It has been a trend for years now. Think of complex applications like personalized product recommendations. Or simpler: latest blog posts, the nearest store, animated countdowns, live tweets around an event. No matter what, the content will always auto-update itself when the recipients opens her newsletter.

This post demonstrates one way to code your own live tweets widget, ready to include them in your emails to promote events, or your channel.

But first things first: How do live tweets fit into the realtime personalization email trend, and why should you care about it anyway?

The case for realtime content in emails

From a technical perspective, the key characteristic of most realtime content technologies in email marketing is that parts of the message are being sent in dynamic image containers. In such an <img> placeholder, the content – i.e. the served image – remains adaptive even after it has been delivered into the inbox. Upon every email open, the graphics might be newly created and switched automatically by the server, depending on the recipients current context and her recent actions. Such subscriber context could be time of day, device, latest behaviour, or current geo location.

From a creative perspective, realtime content is able to prevent a drop in relevance due to outdated content: products that are out of stock or that have already been bought, nearest stores that are meanwhile miles away, supposedly or latest blog posts or tweets that have already been replaced by new ones. Using animated countdowns, counting down to the end of a flash sale or to the beginning of an event, the technology is equally capable of strengthening the call to action.

As a result, subscribers benefit from a better user experience, and marketers increase their subscriber lifetime values.

Live tweets are a classic, however…

Live tweets are one application of realtime email content. Relevant service providers like Kickdynamic, LiveClicker or Movable Ink, to name a few, offer it out of the box.

I love all these and what they brought to the email ecosysten. It’s all darn easy to use. However, one might find things to tweak. Like limited options to customize tweets. For instance, when I first experimented with live tweets in emails, I missed options to exclude specific tweets by, say, tweet ID, screenname and keywords.

My first thought wasn’t about free riders that misuse the feature just to see their – potentially offensive – message appear in a newsletter. It was more about making sure there is a diversity of voices instead of just one channel filling all tweet slots in the image. Which comes pretty natural, because the organizer posts about an event very often during the promotional phase.

Also, I wasn’t too sure whether the commercial offerings were able to break image caches from email providers.

Code your own live tweets

To have full control, and for the fun of it, I created my own experimental live tweets. The webserver, which I use, interprets Python scripts. So I gave it a try. Please find the resulting code below.

If you reuse it, you may have to replace a few things:

  • your tweet query (e.g. “#MyEvent”),
  • your Twitter API credentials (https://apps.twitter.com/),
  • the server paths to the font files (regular & bold) , which are used to render the tweet information,
  • the sever path to the file in which the script keeps a memory of past tweets (“database”),
  • the server path where the script should place the rendered tweets images.

I used a cron job to execute the script every few minutes. It does not overwrite the existing latest tweets image, but places one aside with an added timestamp. This prevents the theoretical case of a subscriber seeing a halfway finalized tweets image, while it is being created by the Python script. The new filename also ensures that Google does not use the cached image.

A PHP script, which is also placed below, chooses the latest image and serves it to the client. You will have to replace the path there, too.

As it’s unusual to refer to an image with a .php extension, I put a a server redirect in place in my .htaccess file, which redirects from .gif to .php. The relevant line reads like

redirect 301 /grafiken/livetweets.gif http://www.emailmarketingtipps.de/php_script_name.php

. This way, I can refer to the image by src=”…/grafiken/livetweets.gif” instead of something like “src=”…/php/livetweets.php”.

###########################
### MODULES & CONSTANTS ###
###########################
import cgitb
cgitb.enable()
import twitter
import PIL
from PIL import ImageFont
from PIL import Image
from PIL import ImageDraw
import pickle
import datetime
import time
import urllib, cStringIO
import glob
import os
import re
import sys
from subprocess import Popen, PIPE


blacklist_ids = [123L, 345L] # Exclude these tweets by ID
blacklist_screen_names = [] #  ["Emarsys", "Emarsys_DE"] # Excude tweets by screen name
blacklist_words_regex = "foobar" # excude tweets by text pattern

tweet_count_query = 100 # maximum number of tweets to query the Twitter api for
tweet_count_max = 5 # maximum number of tweets in a screen shot
tweets_filename = r'mypath/data/twitter_result.pkl' # archive of historic tweets
screen_names_unique = True # whether the tweets image should only include a screen name only once
tweet_query = "#emarsysrevolution OR revolution.emarsys.com -filter:retweets" # the Twitter query
twitter_consumer_key = '..................'
twitter_consumer_secret = '..................'
twitter_token_key = '...................'
twitter_access_token_secret = '...................'

img_width = 540 # the output width of the tweets image
img_mode = "RGB" # the mode of the tweets image (=> fileisze)
line_space = 2 # spacing between tweet text lines in tweets image
margin = (5,5,10,5) # t,r,b,l
color_bg = (255, 255, 255) # background color
color_fg = (0,0,0) # text color normal
color_fg_gray  = (101,119,134) # text color date
font_size = 14
retinafy = True # whether to double the tweets image dimensions for retina displays
if retinafy:
    font_size *= 2
    img_width *= 2
font = ImageFont.truetype("mypath/fonts/arial.ttf", font_size) # regular font
font_bold = ImageFont.truetype("mypath/fonts/arialbd.ttf", font_size) # bold font for screen name
output_path = "mypath/gfx/" # where to store resulting tweet image
file_prefix = "twitter_event_" # filename of a tweets image (followed my timestamp and .png suffix)
#################
### FUNCTIONS ###
#################
def get_tweets(tweet_query, tweet_count_query):      
    try:
        api = twitter.Api(consumer_key = twitter_consumer_key,
                          consumer_secret = twitter_consumer_secret,
                          access_token_key = twitter_token_key,
                          access_token_secret = twitter_access_token_secret,
                          tweet_mode = "extended")
    except:
        return -1, []
   
    try:
        with open(tweets_filename, 'rb') as f:
            result = pickle.load(f)  
    except IOError:
        # Unable to open file? It's probably not there => first time the script is called
        result = []
        result_new = api.GetSearch(term = tweet_query,
                                   count = tweet_count_query,
                                   result_type="recent")
    except:
        return -1, []
    else:
        # File opened => only get new tweets
        result_new = api.GetSearch(term = tweet_query,
                                   count = tweet_count_query,
                                   result_type="recent",
                                   since_id = max([status.id for status in result]))        
       
    if (len(result_new) > 0):
      # if new tweets => append
      result = result_new + result
      with open(tweets_filename, 'wb') as f:
          pickle.dump(result, f)      

    return len(result_new), result




def check_exclude(
        status,
        blacklist_ids = [],
        blacklist_screen_names = [],
        blacklist_words_regex = [],
        screen_names_unique = True):
   if status.id in blacklist_ids or status.user.screen_name in blacklist_screen_names or re.search(blacklist_words_regex, status.full_text) is not None:
       return True
   if screen_names_unique:
       if status.user.screen_name in screen_names:
           return True
   return False



def wrap_text(text, width, font):
    # https://stackoverflow.com/questions/11159990/write-text-to-image-with-max-width-in-pixels-python-pil
    text_lines = []
    text_line = []
    text = text.replace('\n', ' [br] ')
    words = text.split()

    for word in words:
        if word == '[br]':
            text_lines.append(' '.join(text_line))
            text_line = []
            continue
        text_line.append(word)
        w, h = font.getsize(' '.join(text_line))
        if w > width:
            text_line.pop()
            text_lines.append(' '.join(text_line))
            text_line = [word]

    if len(text_line) > 0:
        text_lines.append(' '.join(text_line))

    return text_lines




def get_profileimg(status, retinafy = True, standard_size = 48):
    f = cStringIO.StringIO(urllib.urlopen(status.user.profile_image_url).read())
    try:
        profileimg = Image.open(f)
    except:
        profileimg = False
        profileimg_width = standard_size
        if retinafy:
            profileimg_width *= 2
    else:
        profileimg = profileimg.convert(img_mode)
        if retinafy:
            profileimg = profileimg.resize((standard_size*2, standard_size*2), Image.ANTIALIAS)
        profileimg_width = profileimg.size[0]

    f.close()        
    return profileimg_width, profileimg




def get_image(status):
   # profile image
    profileimg_width, profileimg = get_profileimg(status, retinafy, 48)    
    # tweet creation date formatted
    created_at = datetime.datetime.strptime(status.created_at, '%a %b %d %H:%M:%S +0000 %Y').strftime("%d.%m.%Y, %H:%M Uhr")
    # wrapped tweet text
    text = status.full_text  
    text = wrap_text(text, img_width - margin[1] - margin[3] - profileimg_width, font)
    text.insert(0, "@" + status.user.screen_name + ", " + created_at + ":")
   
    # create tweet image
    text_height = font.getsize(text[0])[1]*len(text) + len(text)*line_space
    img = Image.new(img_mode, (img_width, text_height + margin[0] + margin[2]), color_bg)
    draw = ImageDraw.Draw(img)

    # draw text
    x, y = margin[3] + profileimg_width, margin[0]
    for line in text:
          if y == margin[0]:
              # First line with twitter handle (bold) and date (gray)
              screen_name = "@" + status.user.screen_name + " "
              draw.text( (x,y), screen_name, color_fg, font=font_bold)
              w, h = font_bold.getsize(screen_name)
              draw.text( (x + w, y), "- " + created_at, color_fg_gray, font=font)
              y = y + font_size+line_space
          else:
              # Other lines: tweet text
              draw.text( (x,y), line, color_fg, font=font)
              y = y + font_size+line_space  

    # draw profile img if possible
    if profileimg is not False:
        img.paste(profileimg, (0,0+margin[0]))

    return img    





def combine_images(images):  
    widths, heights = zip(*(i.size for i in images))
   
    total_height = sum(heights)
    max_width = max(widths)
   
    combined_image = Image.new(img_mode, (max_width, total_height))  
   
    y_offset = 0
    for img in images:
      combined_image.paste(img, (0,y_offset))
      y_offset += img.size[1]
     
    return(combined_image)
############
### MAIN ###
############

# ----------------------
# Check for (new) tweets
# ----------------------  
new_tweets, result = get_tweets(tweet_query, tweet_count_query)
if new_tweets == 0:
    print "No new tweets, nothing to do: exiting..."
    raise SystemExit(0)
elif new_tweets < 0:
    print "An error occured during tweets check: exiting..."
    raise SystemExit(1)    
else:
    print "Added", new_tweets, "new tweet(s): updating tweets image..."

# -------------------
# Create Tweets image
# -------------------
images = [] # will contain tweet images
screen_names = [] # will contain screen names from included tweets
for status in result:  
    # skip this tweet?
    if check_exclude(status,
                     blacklist_ids,
                     blacklist_screen_names,
                     blacklist_words_regex,
                     screen_names_unique) is True:
        continue
    # max number of tweets reached?
    if len(images) >= tweet_count_max:
        break

    # turn tweet status object into tweet image
    img = get_image(status)
     
    images.append(img) # add tweet image to list of tweet images
    screen_names.append(status.user.screen_name) # add screen name to check against duplicates when unique screen names is set

image = combine_images(images)

# ----------------------------
# Save Tweets image & clean-up
# ----------------------------
# delete old files
m = glob.glob(output_path  + "/" + file_prefix + "*.png")
if (len(m) > 2):
    m.sort()
    for f in m[0:-1]:
        os.remove(f)    

# save new file: TMP_ + prefix + timestamp + .png
fn = file_prefix + str(int(time.time())) + ".png"
image = image.convert("P") # 8-bit => smaller fileisze
image.save(output_path  + "TMP_" + fn, "PNG", optimize = True)

# after saving completed: remove TMP_ prefix and make it available
os.rename(output_path  + "TMP_" + fn, output_path  + fn)
try:
  os.remove(output_path  + "TMP_" + fn)
except:
  None

print "Saving" + output_path  + fn + "..."
<?php
  // Server right image
  $fns = glob("twitter_event_*.png");
  sort($fns);
  $fn_newest = $fns[sizeof($fns)-1];
  $url = "http://emailmarketingtipps.de/images/" . $fn_newest;
 
  if (headers_sent() === false) {
    header("Cache-Control: no-store, no-cache, must-revalidate, max-age=0");
    header("Cache-Control: post-check=0, pre-check=0", false);
    header("Pragma: no-cache");
    header("Location: ".$url, true, 302);
    die();
  }
?>

 

Here’s the result as of 4th Oct. 2018:

That all being said, the image with the specified width and fonts then looks like this:

And that’s the way it’s meant to be embedded into emails, in order to auto-update itself:

<img src="https://www.emailmarketingtipps.de/gfx/ema_revol_tweets.gif">

(To test it: i) Tweet about #EmarsysRevolution or post a link to revolution.emarsys.com, because that’s what the script catches, ii) wait up to 15 minutes (=current update-interval), iii) then check the image. 🙂 )

Enjoyed this one? Subscribe for my hand-picked list of the best email marketing tips. Get inspiring ideas from international email experts, every Friday: (archive♞)
Yes, I accept the Privacy Policy
Delivery on Fridays, 5 pm CET. You can always unsubscribe.
It's valuable, I promise. Subscribers rate it >8 out of 10 (!) on average.

3 Responses to Enhance your emails with a self-coded live twitter feed

  1. How would I be able to code both Python and PHP together and please let me know about the online environment where I can test and create app.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.