Case study: How a unified approach to AI-copy creation increases CTOR by 41.34%

By utilizing AI — ChatGPT 3.5, to be exact — for smart modules, we managed to automate communication with AI to make email production even more time-effective. And what’s more important, we learned to make AI generate unified texts across the modules/content units.

As a cherry on top, it all resulted in higher CTOR.

The challenge, or the goal of the experiment

Email product cards and blog post annotations usually include three text elements: CTA buttons, titles, and descriptions. If AI generates these elements, they might seem unconnected and use varied terms as AI processes each element individually. But titles and descriptions need to match in terminology and the message.

Our goal was to make AI generate consistent texts across modules/content units.

Solution   

• as many of you may know, we designed the Smart Elements feature a few years ago. This feature enables you to create numerous product cards and email digests with a single click. How does it function? You design your product cards once and save them as a module. Then you paste a link to your product page or blog post, and Stripo extracts all the required information from those pages, inserting it directly into your email;
• also, we recently announced our integration with ChatGPT 3.5, a powerful tool that facilitates the generation and optimization of copy in your emails. This eliminates the need for third-party tools or manual copying and pasting of content.

So we combined these two features in order to figure out a way to “force” AI to generate a unified copy for the entire content units. 

Important to note

Currently, Stripo allows you to use AI in the text blocks and for subject lines, but if AI performs well for the smart modules, we will make it available to everyone. The AB test will show if our clients can benefit from this new feature.

You can also use our embedded AI tool for generating texts. Create new texts, improve the existing ones, change the tone of voice, fix grammar mistakes, or add relevant emojis directly in the editor. You can find a detailed guide in our blog article.  

Results

We’ll talk about both the technical part and email performance.

The technical part

We took three steps to combine AI and smart modules:

Step 1

We designed a smart module for a blog post annotation.

Step 2

We had to create a prompt for ChatGPT so that it scans the link and title for optimizing the product/blog post title and coming up with the best description. We tried a lot of prompts and conducted a lot of internal tests to find the best prompt. So, we can say that writing a good prompt is an art. 🙂

Here’s the prompt we found the most effective and used for each module in our digest:

Imagine you are an email marketing copywriter and need to help your team write some elements for the email they will send to customers. Your colleagues send you a link based on which you have to provide them with the necessary elements of the text. You are using all the best practices in copywriting and email marketing to deliver great results.

This can be a blog post, product page, or email template design page. 

You can use, if necessary, emoji (in the title only, 1 per title).

Here are some limitations you have to follow:

• title less than 75 characters;
• the description is between 150-200 symbols;
• CTA — 2-3 words maximum;
• if the title of content inside the webpage contains any numbers, like “10 examples”, “6 rules”, “5 frameworks,” etc, you must use this number inside the title element without changing the number itself and with the number only. The title on the webpage is marked as H1 HTML markup;
• if the same link is provided to you several times — you need to get different results each time;
• do not use quotation marks anywhere.

Temperature: 0.7

Top_P: 0.6

Frameworks to use:

• social proof;
• benefit driven;
• question-based;
• the reasons why;
• the how-to framework.

All elements should be done in a friendly tone of voice. 

When the link is provided, you should open it and, based on the content, write 1 title, description, and CTA in a single (one table for all elements and frameworks) table format using a few different frameworks, where elements on the horizontal axis and frameworks on the vertical.

If it's a blog post about a specific topic or with examples, please use the general idea without creating an email based on inner examples. For example, if you post about examples in teaser emails, use only the general idea on this blog post, not one of the specific examples.

If you are ready, please type in the chat, “Gimme a link, please.”

If everything is ok, ask for the next link.

Step 3

We had to teach AI to put the necessary pieces of information into respective fields.

We succeeded here.

This is the email with the copy written by a human

This is the email with AI-generated content

Here’s what the combination of generative AI and the smart modules feature gave us:

• AI generates copy for three different elements inside one module at once, so this cuts the number of clicks we make and saves us time;
• AI generates a unified copy.

Email performance

As you can likely infer from the blog post title and the numbers we shared above, the results are pretty satisfying.

To ensure the results were truly a consequence of the changes we made to the emails, we conducted four separate A/B tests, each involving a different email digest.

Numbers themselves

Every time we chose a different group of recipients (about 84,000 each), whom we later segmented:

• 50% of the subscribers received the email digest written by a human;
• 50% of each group received an AI-generated email.

The results of the four email digests are similar, so we’ll now describe the most recent AB test.

Please be advised that we used the same subject line for both email versions.

Human-written email

• recipients — 42,024;
• OR — 31.46%, i.e. 13,221 people opened the email;
• CTOR — 5.72%, i.e. 756 people clicked the links in the email and visited our blog to continue reading.

Snippet of the email, click map

AI-generated email

• recipients — 42,078;
• OR — 31.52%, i.e. 13,264 people opened the email;
• CTOR — 8.08%, i.e. 1072 people clicked the links to visit our blog.

Snippet of the email, click map

Wrapping up

We have AB-tested how AI works for email digests and how you can automate email copy creation and optimization. And we’ve come up with the following conclusions:

1. Generative AI performs well if you use it right, even for complex things, such as entire modules.
2. It is possible to keep the content consistent across these modules and entire emails without human interference. Yet, you can always check, optimize, and modify AI-generated copy. Just to let you know, you can use modules for your future email campaigns. 
3. You can automate email copy creation.

Given that we greatly enjoyed the results, we're releasing the generative AI for smart modules feature. Expect the release to be on June 16th.