Design Marketing
~ 12 min read
1118 views
rate it
23 April

Email A/B Testing: Tools, Techniques, and Best Practices

Stripo / Blog / Email A/B Testing: Tools, Techniques, and Best Practices

Making a good newsletter is not an easy thing. It is important to take into account the subject line, color, font style, content, images, videos, and more. Even a smiley face can affect the outcome. Relying on your taste only is obviously a way to lose. So, you need to go the other way. Email A/B testing comes to the rescue.

In this article, you will learn how to run email AB tests, the complexities of A/B testing, and the benefits of this method. 

What is A/B testing in email marketing?

The market is changing, customers want something new, and the business is adapting to these needs.  A/B testing, or split testing, is used precisely to detect the mood of people that read emails.

The pros of split testing your email campaigns

  • You will understand your customers better.

And will learn what they really want. Even if you are experienced in business or marketing, your assumptions may not always match the reality. Relying on your own beliefs is good, but it’s better when the test results prove your beliefs are correct.

  • You’ll reduce the risks associated with the changes.

If customers don't like the changes, part of the audience may leave. Consult with users through A/B testing for email marketing.

  • You’ll improve email conversion.

Doubts about the design of the email, subject lines, products and their location in emails, the tone of communication, and more can be resolved through the AB testing.

What metrics to check?

An email marketer has to constantly analyze data to improve the situation. Below we will share three metrics that will help you track the effectiveness of your email campaigns.

AB Testing_Measuring Metrics of Email Campaigns

Open Rate (OR)

One of the key metrics. It shows how many times your email has been opened. For example, an email was successfully delivered to 100 recipients and was opened by 60. What is the Open Rate in this case?

This metric’s formula:

Open rate = number of unique opens/ number of successfully delivered emails* 100%

Click-through rate (CTR)

It shows how many times recipients clicked on links in emails. Continuing our example above, let's imagine that your email was received by 100 people, and only 25 clicked on the links in it. The CTR of this email is 25%. If the click-through rate is low, it means your email strategy has some mistakes. In this case, it is necessary to analyze, and fix the them.

This metric’s formula:

CTR = (number of clicks / number of delivered emails) * 100%

Click-through rate (CTOR)

CTOR is an indicator of engagement. Imagine that your email was delivered to 100 people, 60 people opened it, and only 10 clicked on the links. The CTOR of this email is 16.6%.

This metric’s formula:

CTOR = number of unique clicks / OR * 100%

What to check with AB testing?

1. Email subject lines

The subject is the first thing a user sees when they find an email in their inboxes. It depends on the subject line whether the recipient will open the email, or will delete it even without reading. We recommend that you pay close attention to the wording and analyze whether the subject line shows the main message, any call to action, or at least drives some intrigue.

AB Testing_Email Subject Lines

2. Email text length

If a user sees a long read when opening the email, his or her desire to read it disappears. Perhaps the information that you want to deliver to recipients is truly useful. In this case, try to delete 1-2 paragraphs or shorten some sentences. You can also use lists that make it easier to perceive the text. 

AB Tests_Bulleted Lists

(Source: Email from Moosend)

How to add lists with Stripo:

  • highlight a necessary abstract of your text;

  • on the toolbar, click the “List” button;

  • pick a necessary type of list.

AB Testing_ENG_Adding Lists to Emails with Stripo

Add bulleted and numbered lists to emails
Join Stripo

3. Email design

Decide what color scheme you’ll use for your email. Check out how the color palette affects a person. For example, Wikipedia says that red creates a sense of excitement and blue creates a sense of reliability? This information will help you incorporate the right shades when designing your email, and thereby evoke the necessary emotions in a person.

But how exactly will the “Right” color “work” on your subscribers? That’s the question the AB testing can help you answer.

3.1. Visual elements in emails

Visuals are an incredibly important part of an email. As you know, most people perceive images better than text. In your messages, you can use pictures, videos, GIFs, etc.

AB Testing_Visuals in Emails

(Source: Email from Stripo)

3.2. Email call-to-action buttons

With just one click, a person can go to the site, to fill out a questionnaire, place an order or write a review about a product or service. This is convenient for the recipient and visually looks better than just a link to the page.

Experiment with its location and design to see which option works best for your company. 

AB Testing_Visual Elements_CTA Buttons

(Source: Email from Nike)

It is worth mentioning that the button’s name should be clear and concise.

For more information on the CTA buttons and ways to build them please refer to our dedicated blog post.

14 min read

56985 views 4.2

Email Call to Action Best Practices Read more

3.3. Email fonts

Of course, there are lots of different fonts. But we want to remind you that some decorative fonts make texts illegible.

AB Tests_Choosing Legible Fonts

Besides, it is also important to stick to the Email accessibility best practices. Meaning, avoid italic, put dots at the end of sentences and lists.

4. Tone of the email message

If you haven't created a tone-of-voice for your company yet, then it's time to do it right now! Detect in what tone you’ll communicate with recipients. Whether it's a simple, friendly tone of speech or the opposite ー formal and businesslike ー A / B testing will help you choose!

An example of a formal speech.

AB Testing_Formal Tone of Speech

(Source: Email from Adweek)

An example of a friendly, cheerful speech.

AB Testing_Friendly Communication

(Source: Email from Chubbies)

 

5. Personalization

Here you can start with addressing customers by names emails.

AB Tests_Personalization in Emails_Addressing by Names

(Source: Email from ProductPlan)

If you know your recipients’ interests, hobbies or wishes — use this information.

For more information on ways to get the most out of email personalization in emails, please refer to our “Personalization” blog post.

6. Send time

Friday is said to be the best day to send entertainment/promo emails. But business emails are to be sent on Tuesday when a person is mentally ready to perform new tasks. But is it relevant for your business? Conduct A/B testing to find out what day/time is the most effective for your emails.

How to Conduct Email A/B Testing?

Let's figure out how to conduct it step by step:

Step 1. Choose a target for your test.

Recklessly changing elements in email, hoping to get a million dollars is not something we should do. Decide what results (what open rate, ctor, etc.) you’re going to consider successful. When you have a particular goal in mind, the results will be way more impressive.

Sometimes, instead of metrics, we need to find out how a new element will work. 

Our team in February 2020, embedded an AMP form in an email. The form allowed users to leave comments/share feedback directly in their inboxes. We wanted to check which option would collect more reviews: a regular third-party form, the link to which we gave in the email, or the embedded AMP form. The second one turned out to be 5 times more effective than the usual form on the site. This encouraged us to create the AMP block "Form", and now our users can also create a similar form for their newsletters.

AB Testing_Leaving Feedback in Emails_AMP Form

By adding interactive forms in emails, you improve your chances to find out what users think about your product and services Add form

Step 2. Select a variable.

First, decide what you will be testing. For the test result to be as effective as possible, test one variable in one email at a time. For example, change the subject line. Even small changes, like emoticons, can lead to high results.

AB Testing_Emojis in Subject Lines

Step 3. Make a copy of the email.

After that, make changes to the variable in the copy of the email. Change the color, font, or a CTA. But remember, just 1 variable at a time.

Step 4. Segment your contact base.

Do it yourself or automatically in your ESP.

Step 5. Process the results.

This is the only way to understand how effective the split test was and in which direction you should move on. It is very important that you compare the results of the two groups over the same period of time. This improves the accuracy of the split testing.

6. Act.

Judging by the results, decide on the next steps. You may need to do a few more tests to get a picture of your perfect email.

Make changes, test, analyze — after all, this is the only way to improve email conversion, increase self-confidence, and earn your customers’ trust.

Tools for A/B email testing

Now’s the time to take a look at the tools :

1. For the subject line.

You already know the importance of the subject line. But how do you evaluate it? by using the Email Subject Line Grader by Net Atlantic.

Tools for AB Testing

2. For email design.

We have already discussed what visuals to test.

You can create all of them in Stripo.

With us, you do not need to create a new email from scratch. Having created the first email for your tests, you simply copy it and edit the necessary elements in it.

AB Testing_Tools for AB Testing

Build emails for your AB tests faster
Join Stripo

3. For detecting the tone of speech

Some are still looking for their unique communication style. And some just want to try something new.

How do you detect which style you have? How do you detect how you sound?

Grammarly app shows over 10 different tones of speech.

AB Testing_Tone Detector_Grammarly

4. For the best send time.

Any modern ESP is supposed to save and analyze the time when the email is opened. Thanks to such tests, you will be able to send emails at a convenient time for recipients.

If you are now looking for a new ESP, you may find our review of the TOP 10 ESPs useful.

Final thoughts

With the help of a split test, you will finally be able to understand which elements of your marketing campaign require special attention and improvement. Follow the rules from our article, use Stripo, and improve your emails.

Build high-quality emails fast with Stripo
Be always wide awake in recent email marketing news, guides, articles and how-to’s