MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

Editor’s Note: In a recent interview with MarketingSherpa Editor Sean Donahue, Research Analyst Corey Trent outlined errors even experienced email marketers make when conducting tests. (My personal favorite – #5). We thought this information was valuable, and wanted to share it right here on the blog for those who do not have a MarketingSherpa membership. Special thanks to our sister company for allowing us to republish the below article…

SUMMARY: Before you conduct your next email test, make sure you’re not falling into a trap that can muddy your results or limit the gains you might otherwise achieve.

We spoke with an email testing expert from our sister company, MarketingExperiments, to uncover common mistakes marketers make when running email tests. Read why good analytics and segmentation are crucial forerunners to testing, and why a blockbuster discovery from one test actually can be a risky thing for a marketing team.

by Sean Donahue, Editor, MarketingSherpa

Testing is an essential component of a strong email marketing strategy. But only if the tests are conducted and analyzed properly to ensure you’re helping – not hurting – your email performance.

“There is a cost for bad testing,” says Corey Trent, Research Analyst, MarketingExperiments. “Bad assumptions based on bad tests can cost you a lot of money and cause you to lose out on a lot of business.”

Trent routinely conducts email tests as a member of the MarketingExperiments sales and marketing optimization research team. Through this work, he’s seen how mistakes, misconceptions and simple oversights can derail a well-meaning marketer’s testing strategy.

We asked him to share his advice for avoiding testing pitfalls, so you can achieve your goal of improving email performance. Here are seven common mistakes he’s observed:

Mistake #1. Not having the right analytics behind your tests

Sometimes marketers must recognize they are just not ready for email testing, says Trent.

In particular, he says, marketers often don’t have the processes or the analytics systems in place to capture a complete data picture — you must be able to tie your email metrics to business goals, such as revenue.

For example, if you’re only measuring email metrics, you might test a new subject line treatment that increases open rates by a few percentage points. Sounds exciting, but without tying subsequent clicks from those messages to website sales, you might not notice that you’re actually decreasing conversions and total revenues because the subject line is creating the wrong expectations about your message.

The good news is that it is now relatively easy to track email activity through to website metrics using tools such as Google Analytics. This program offers marketers ways to track email messages through specific goals on your site, such as product pages viewed or time spent on site (see Related Resources below).

“Email tracking all the way through to business goals is available for everyone, and doesn’t take a long time to set up or require having an IT team to do a lot of programming for you,” says Trent.

So make sure your analytics systems are robust enough to support your objectives before you begin email testing.

Mistake #2. Not segmenting email lists for tests

Many marketers have large email databases, but don’t know a lot about the records held within. In these cases, they may conduct an email test using their entire database – and create a muddy results picture.

“Without segmented lists you don’t get good test results,” says Trent. “You get all these people responding differently to your emails, which pulls your results in all different directions.”

Spend the time to segment your database and understand the different characteristics of the segments before you embark on email testing. The more you know about unique segments within your database prior to testing, the better chance you’ll have of finding the right messages to appeal to them.

Mistake #3. Stopping tests after one big win

“With email more than anything, we see people get a big win and stop testing,” says Trent.

As exciting as those big wins may be, they shouldn’t be the end of your testing process. The makeup of your email lists is constantly changing; external factors, such as the economy, also impact subscriber behavior; and your competitors’ campaigns and tactics are always changing as well.

This constant state of change means you must routinely work on the messaging, layout, calls-to-action and other elements of your email messages to ensure you’re getting the full benefit of a testing program.

Mistake #4. Testing too often

The flip side of stopping tests too quickly is falling into a pattern of repeated testing, to the point that you simply can’t keep up with all the data you’re generating.

When finding the right pace for testing new email treatments, think critically about the tests you want to perform and their potential impact. Focus on the ones that have the greatest potential impact on your specific business goals.

Trent suggests researching your current email strengths and weaknesses: Examine your own email and website metrics and study industry research to benchmark your own performance against your industry peers. This process can uncover where your email programs are delivering weak performance – and where you have a bigger chance of improving your company’s bottom line through testing.

“Those deep dives into the data you have available help you prioritize that list of potential tests,” says Trent.

Mistake #5. Overlooking email copy tests

Today’s email messages are so versatile that marketers have a wealth of options to test: Layout changes, images, fonts, colors, social media integration and so on.

But the ability to tweak these elements can make marketers forget about the old standby: copy tests.

“What often gets left out is how important the copy and communication pieces of an email are,” says Trent.

Copy changes can deliver surprising results, so keep them on your list of test options. You can try using a different voice for your message, long copy vs. short, different copy layouts, or even a personal letter from someone in your company.

Mistake #6. Always testing additions, rather than subtractions

The pressure to generate more engagement with subscribers can push marketers to continually test new additions to their emails: More images, more buttons, more links or more colorful text.

Too many additions can actually confuse recipients, making it hard for them to decide what to do with your message.

Instead, Trent has seen marketers achieve strong performance gains by eliminating elements from an email and reducing the number of decisions a recipient has to make.

By focusing a message on a single goal — such as inviting recipients to an event or generating sales within a specific product category — you can see whether there are extraneous elements that might be distracting recipients from the desired action. You can then test the impact of removing those distractions.

Mistake #7. Not asking other customer-facing teams for test ideas

Great test ideas don’t always come from the marketing team. It’s likely you have other sources within your company whose insights can shape a good email test.

Trent recommends talking to sales people or customer service representatives to learn what they’re hearing from customers. The feedback these teams receive from customers about your products and services — or why they chose to buy from you — can spark ideas for new subject lines, email copy, or specific products to feature in email promotions.

“Don’t be scared to go down and talk to those people to get some good ideas.”

Related Resources

The Magical Metrics Tour: Demystifying the secrets behind analytical “tricks” to help you drive ROI (describes how to track email clicks with Google Analytics)

What Else Can I Test….To Increase Email Click-through?

Discover the Best Time to Send Email: 4 Test Ideas (for MarketingSherpa members)

Improve Your Email Programs: 5 Test Ideas (for MarketingSherpa members)

Email Marketing, Research Topics

Email testing produces some of the most interesting results I see here at MarketingExperiments. The cause for this is a combination of constantly changing variables.

For one, content within email tends to change more often than your typical landing page. This makes optimizing for content more challenging as different topics are likely to garner different levels of interest from the segments within your email list. So results will change each month based on the content alone – making A/B testing the only reliable method for measuring progress.

In addition, email lists themselves prove to be a challenge, as what works for one list may not work for another list. Even within lists, especially aggregated lists, you will see different results based on the value proposition, content, layout, and calls-to-action (CTAs) used in your email.

And to further complicate matters, you are still dealing with a funnel process in which your email must first reach a user (avoiding spam filters, personal filters, etc.), your subject line must interest the user enough to open the email, your email must display properly (with images on and off) and be compelling enough to achieve a click-through to your landing page where the battle for a conversion wages on.

In today’s world of overloaded email boxes, people declaring email bankruptcy, spam filters and everything else, this game is only getting more difficult – for marketers and users alike.

With that said, I’d like to offer up my own favorite email testing tricks and tips. It’s important to understand that what works for one segment, list, or industry will not necessarily work for another. In fact what works one month for a list may not work next month. It’s an ever-evolving process in which you must always challenge your own best practices to maximize your results.

1. Text-only email

This is my silver bullet of email optimization. In a world where everyone wants their logo, business cards, websites, and emails to be as shiny and pretty as possible, it’s easy to forget that email is a text-based medium.

Out of all the emails you actually read each day, how many are HTML vs. text-only? The important emails you receive each day – the ones from your family, friends, and co-workers – are all likely text-based. This means the HTML emails you receive are most likely not from one these aforementioned groups and thus likely less important to you on a personal level.

Obviously there are caveats to this – such as an email from your bank or a Facebook friend request – but the truth is, when you send an HTML email you are already fighting a certain level of banner blindness. If you currently only send out an HTML email, I’d challenge you to A/B split test against a text-only version of your email and measure the results.

2. Story format

Emails are a form of value exchange. In exchange for someone’s time and interest, you must first provide something of value.

One of the easiest, most interesting ways to provide value in an email is to tell a relevant story. Not only can telling a story create interest in your topic, but it also can provide you with a natural sounding CTA of “Continue Reading” or “Read More” that requires less commitment from the user before clicking through to your landing page.

The downside to the story format is it requires you to have an interesting story to tell as well as an audience willing to read through the text.

3. Big button

Sometimes if your offer is compelling enough, all you need to do is give people a place to click. It helps if your email list is already familiar with your brand, is interested and familiar with what you’re offering and needs little explanation before clicking through.

For example, our Marketing Director is drawn to J. Crew’s buttons like a moth to a lightbulb…

The focus should be on the value proposition and the CTA copy as this will determine the commitment level required to achieve a click-through. A “Learn More” CTA will bring more clicks, but less qualified traffic than a “Buy Now” CTA. Experiment with this CTA copy to see what works best for your particular audience and dial in the right amount of click-through vs. conversion.

4. Multiple CTAs

I’ve found that when dealing with large, aggregated email lists, including multiple, different CTAs can help increase click-through on the basis that different people will be interested in different aspects of a topic.

For instance, if you are offering a free PDF download on a topic that you hope will encourage people to provide their email address, and you also have a webinar that does the same thing, go ahead and include a link to “View the PDF” and another CTA to “Sign-up for the Webinar.”

What you’ll find is that offering multiple contrasting calls to action will result in a higher click-through rate as some people will be interested in downloading the PDF, while others will prefer the webinar – two different mediums that attract two different groups of people but achieve the same objective.

5. Digest

If you currently send out a longer email newsletter, I’d encourage you to try a digest format. In the digest format you offer several links at the top of the email that either direct users to a particular article on your website or anchor down to an excerpt within the email itself, followed by a CTA to “read more.”

I’ve found digest emails work well when you have a lot of content to offer and you are able to effectively arrange this content so as to attract a wide variety of clicks. It’s a combination of the story format and multiple CTA emails mentioned previously.

Related Resources

The Five Best Ways to Optimize Email Response: How to craft effective email messages that drive customers to action

Optimize your Email in Three Steps: How one marketer tripled revenue from their house list

Order your custom Email Response Optimization Package

Email Marketing, Research Topics

Editor’s Note: The MarketingExperiments community is an interactive group with a great deal of questions and answers between marketers and their peers as well as with the MarketingExperiments staff. Occasionally we publish these interactions on the blog when we think there is a particularly good question that our readers can benefit from…

QUESTION:

Hi, I completed the MEC Email Certification course a while back. I misplaced the MEC optimization formula. I want to share it with some internal people.  Can you please send me the formula?

Thanks,

Karen
Customer Communications Manager
Cleveland, Ohio

ANSWER:

Ahhh, yes. You’re probably thinking about the “Optimization Sequence,” which applies to all channels.  See if this looks familiar…

MarketingExperiments Optimization Sequence:

… meaning that when approaching an optimization initiative, you should first optimize the product factors of your offer – ensure you have the best product available, for at least one significant, describable customer segment.

Only after doing that should you optimize the presentation factors of your offer – ensure you have the best, most compelling offer value proposition available, for at least one significant, identifiable customer segment.

You do this by applying the relevant conversion heuristic (e.g., for a landing page: C=4m+3v+2(i-f)-2a; for an email offer: eme= rv(of+ i) – (f + a) ).

Only once you have confirmed a reasonable level of optimality of both the product and presentation factors of your offer should you embark on optimizing the channel factors – driving as much profitable demand to your optimized conversion process as you can.

This is done by channel identification, selection and optimization, using techniques such as channel mapping, paid search optimization, SEO, affiliate blueprinting, etc.

All the best,

Bob Kemper
Director of Sciences
MECLABS Group, LLC

Related Resources

Optimizing Your Landing Pages

Email Optimization

Optimizing Offer Pages

Internet Marketing Strategy

It’s a little over-simplified, but an email marketer’s job is to get the right message to the right person at the right time to achieve a specific goal. Doing that means understanding what motivates subscribers to open a message and engage with your offer – and that’s where the process gets tricky.

Like our colleagues at MarketingExperiments, we at MarketingSherpa believe that nothing provides the better insights into the “right” approach than a good test. A marketer’s personal bias, best guess, gut instinct or assumptions aren’t enough. In fact, they’re often wrong. You have to be willing to let your audience SHOW you what motivates them.

Today in Munich, MarketingSherpa is hosting its second Germany Email Marketing Summit, which features a Case Study that demonstrates the power of testing to determine customer motivation. VNR.de, a publisher of lifestyle and professional advice from experts in their fields, is sharing the results of a list-cleansing/subscriber reactivation campaign they recently conducted.

Winning back “inactive” subscribers

The campaign targeted “inactive” members of their list, which they defined as subscribers that had not opened or clicked an email in 120 days. They wanted to either reactivate those subscribers, or else determine that they were truly inactive and remove them from the list. So they set up a four-message reactivation campaign to encourage a response.

Each message took a different approach to the reactivation effort:

- The first was a survey about email preferences
- The second was a request for subscribers to update their personal information
- The third was a contest to win a book
- The fourth repeated the request to update personal information

What is more appealing than FREE?

Going into the campaign, the team believed the contest offer would have the best response. After all, people like getting free stuff, right?

Maybe not: The contest offer had the weakest open rate and clickthrough rates of the four messages. Its open rate was 60% lower than the best-performing email – the survey about email preferences. And the contest offer’s CTR was 82% lower than the best-performing email.

The good news is that the reactivation campaign was a success overall. They reactivated 9% of the inactive subscribers they targeted – and they won a MarketingSherpa Email Marketing Award for it.

They also learned important lessons about what motivates their subscribers. Their conclusion: “People seem to be most interested when we are interested in them.”

Final lesson: Assumptions are no match for results data. So get testing!

Sean Donahue is the editor of MarketingSherpa, a research firm publishing Case Studies, benchmark data, and how-to information read by hundreds of thousands of advertising, marketing, and PR professionals every week.

Email Marketing, Research Topics

I’ll admit that I am a Twitter novice. Compared to social media gurus, some of whom have tremendous experience with the platform (up to two* years!), I am still very much in the learning-by-doing phase. Then again, aren’t we all?

As I try to be informative and give back to the Twittersphere, one of my email-related tweets was picked up by a Florida marketing agency that services several metros nationwide. With our Email Optimization clinic series underway, I was more than happy to provide an analysis of a broad-spectrum campaign that they had planned. Luann, their president, was as excited as I was about making a Twitter connection.

With Luann’s permission, I wanted to share my thoughts and recommendations with our readers. Here is an edited copy of the email response that I sent to her:

Hi Luann,

(click image to zoom)

Here are a few thoughts based on the email message creative I got from Noele, along with the requisite assumptions I’ve made. I hope they will be helpful.

There are two important caveats:

1. I don’t believe in best practices. Everything I recommend is normally tested until I find out what really works for the particular product and customer segment.
2. I want to be as helpful as possible, so I am not pulling any punches; the comments below are not a reflection on your company’s competence or reputation—just how they are communicated via this email message.

The fundamentals: Optimizing thought sequences

In optimization, our objective is not to create better design or copy. Our objective is to affect different thought sequences, and design and copy are our tools. A useful way to examine the thought sequences we need to address is through three simple questions that arise in the mind of the email recipient immediately, whether consciously or unconsciously:

1. Who is sending me this email?
2. What is it asking me to do?
3. Why should I do it?

Our job is to answer these questions as directly and quickly as possible using copy, graphical elements, and layout of the email.

Without specific information about your list, I am going to assume (based on email content) that it contains a large segment that has never done business with your company and perhaps has never heard of it.

Communicating Efficiently: Make it an easy read

The body of the email appears singularly focused on its graphic design and a clever visual way to represent what you do. I suspect that your target customers would prefer a plain-English explanation instead.

They would also likely appreciate it being summarized into a strong, benefits-focused headline, supported with several key reasons why they should use your company’s services, rather than your competitors’.

How it appeared in my Outlook preview pane. (click image to zoom)

I am making an assumption about your target customer segment(s), but from my experience—especially with B2B—black text on a white background works best most of the time. There’s rarely a better way to communicate with busy professionals.

Relying primarily on text, rather than images, will likely work better for you because in default Outlook setup with a preview pane, most people will see blank white boxes instead of your message—and promptly delete it. Alt text helps, but not as much as well-formatted HTML text. You need to make sure that your email degrades gracefully: it needs to read acceptably with images turned off and in plain text.

Communicating Value: Make it clear why you are the best choice

Again, there is no real headline here. The question “Is your business missing something?” is so generic that I can’t imagine it being compelling at all. You can have a successful question-format headline, but it needs to point to a specific problem that you know your customer has.

A great way further to support your value proposition is by telling the reader what your customers say about you. It’s more powerful than anything you say yourself.

There is another challenge with communicating value: you are offering a range of very different services. Sent to a large enough list, this will get you calls, but I would invest some time into 1) trying to segment your list and offer only the most relevant services to each segment, and 2) if you can’t segment or still end up with a large “general” segment, help your reader understand which service is right for them.

Communicating Action: Make it clear what to do next

You don’t want to leave this up to the recipients to figure out. That’s what we call “unsupervised thinking.” You need to do most of the work for them—or you won’t get the click.

There is no clear next step. Here’s what I can picture a recipient thinking: “It looks like you just want me to sign up for the newsletter. It’s the biggest CTA (call to action). But I don’t know who you are. I really don’t care about getting latest news postings on your website. If we already have a relationship, why am I getting this generic email?”

In the end, you are not giving the reader a specific reason to contact you. This goes back to building the problem, explaining why you are the best solution, and telling the reader what they’ll get by clicking where you want them to click.

If this is an email to an unsegmented list, I suggest two options to test:

1. Have only one CTA (you can repeat it at the top and at the bottom, but ultimately you should be asking them to do one thing). The job of this email will be to build enough confidence/interest in your company to get a click. Then you can provide options (if relevant) on the landing page.
2. Have several distinct offers, making very clear which one applies to which customer segment or specific problem it’s solving (even if you can’t segment the list, you should know what the key segments are). Then the job of this email is to help the reader quickly decide which offer is most relevant, and click on the corresponding CTA.

I hope these insights will be helpful, and I look forward to hearing about the results you were able to achieve with them.

Sincerely,

Boris Grinkot

To see more email optimization ideas, you can listen to the replay of our last live web clinic, where the MarketingExperiments team offered testing ideas for audience-submitted email marketing messages.

* I’m not counting 2007—come on!

Email Marketing, Practical Application, Research Topics

Editor’s Note: The MarketingExperiments community is an interactive group with a great deal of questions and answers between marketers and their peers as well as with the MarketingExperiments staff. Occasionally we publish these interactions on the blog when we think there is a particularly good question that our readers can benefit from…

QUESTION:

I recently watched The Five Best Ways to Optimise Email Response seminar by Dr Flint McGlaughlin. I found it extremely enlightening and it provided a lot of food for thought. However, I have a quick question with regards to slide no. 22.

I appreciate your time and I’m sure you receive plenty of mailings of this nature; therefore I will get straight to the point.

In this slide, the recommendation is to change the subject line of the mailing from “Thank You For Making Us Your Florist Of Choice” to “15% Off – Our Way Of Saying Thank You!”

I understand why the wording would be changed to make it more endearing to the receiver but I wondered if the symbols added would increase the risk of the mailing being filtered and more inclined to be highlighted as spam – therefore reducing the success of the mailing.

In my experience I steer clear of any symbols in the subject line when sending large mail shots, especially %, ! and £. Am I being too cautious?

Kind regards,

Chris, BA(hons) Business & Marketing
Marketing
London

ANSWER:

Hi, Chris. Thanks for your question.

If I might broaden the question slightly to interpret its essence as a transferrable principle, could I restate it as…

How much validity is there to the conventional wisdom that, in the Subject Line of an offer email message, numbers, certain symbols (especially £/€/$, %, and !) and “SPAM words” such as “Free” and “discount” will cause a dramatic reduction in deliverability, and consequently effectiveness?

… if so, then it’s surely an important one.

In the case of the particular company and study referred to on Slide 22 – that was precisely one of the questions we set out to answer.

What you couldn’t see in the context of Dr. McGlaughlin’s presentation at the MarketingSherpa Email Summit in Miami is that this particular two-treatment comparative vignette was just a tiny part of a much larger and broader study. We intended to test the specific, widely accepted presumption you mentioned.

We were also exploring a host of other best practices to see how valid they remained through the evolution of regulations as well technical filter changes by email service providers (ESPs) since the time they were first introduced and anecdotally adopted (around 2003-2005).

This was important because we know from our foundational Offer/Response-Optimization principles of “clarity trumps persuasion” and “specificity converts,” that the clearer and more specific subject line – i.e., the one with the “15% Off…” copy – should convert better.

What we found was that there was, in fact, a small but significant difference in deliverability – interestingly, it was more pronounced among the smaller ESPs. In addition, as we had predicted based on the “eme” heuristic, the Open Rate actually declined (…by more than 25%).

In the end, though, the central research question was “Which email subject line will result in the greatest projected net revenue?” As revealed in Dr. McGlaughlin’s presentation, despite the slight dip in Delivery Rate, and the (what would otherwise have been alarming) drop in Open Rate, the Click-through Rate (CTR) to the landing page was 60.3% higher.

What he may not have mentioned is that, in direct answer to the research question, the Treatment subject line yielded a 56% increase in projected net revenue vs. the Control.

So, while it appears there is still at least some validity to the commonly held belief that special characters in the email Subject Line reduces deliverability, our research (this experiment plus two others conducted with different products and industries) suggests that when they serve to do so, these negative factors are dwarfed by the power of clarity.

I hope that’s helpful, Chris.

All the best,

Bob Kemper
Director of Sciences
MECLABS Group, LLC

Dr. McGlaughlin will next be teaching and speaking about email marketing at MarketingSherpa Email Marketing Germany 2010 in Munich on March 8^th and 9^th.

Dr. McGlaughlin’s four-hour workshop and keynote presentation will cover email capture rate and quality, open rates, conversion, and building customer trust and loyalty with email. He will also be conducting live optimization of audience submissions – a lively and always-popular segment.

Email Marketing, Research Topics