Shelby Dorsey

The Importance of Testing: How one test applied to two email sends resulted in different audience responses

November 23rd, 2015

At MarketingExperiments, sister company of MarketingSherpa and parent company of MECLABS, we believe in testing. Our Web clinics are a testament to this belief – every month we share research that is designed to help marketers do their jobs better.

This culture of testing encouraged me to run my own test.

First, I ran the test on a MarketingSherpa send. After the results of that test came back, the same test was then applied to a MarketingExperiments’ newsletter.

By running virtually the same test twice for two different audiences not only did I learn more about the preferences of ourreaders, but I also learned how incredibly important it is to test, even when you are sure you know what the results will be.


The MarketingSherpa test

As the copy editor at MECLABS, I get to see all of the copy produced by both MarketingExperiments and MarketingSherpa. One of my responsibilities is overseeing the email newsletter sends. Every Monday, MarketingSherpa sends out a Best of the Week newsletter which features the most popular articles of the previous week and introduces the new book for the MarketingSherpa Book Giveaway.

The copy in these newsletters was extensive. Every article listed had a full summary which, as a reader, I imagined would seem overwhelming when opening this email on a Monday morning.

With the help of Selena Blue, Manager of Editoral Content, and Daniel Burstein, Director of Editorial Content, I decided to change reduce the length of the Best of the Week newsletter and use an A/B split test to determine which format our readers preferred.

Below is a side-by-side of the Control and the Treatment. 


After running this test for six weeks, there was a definite trend of higher total and unique clickthrough happening for the shorter treatment versions. My hypothesis proved to be true — less copy in the email meant higher engagement rates with readers.


The MarketingExperiments’ test

Based on the success of the MarketingSherpa test, I thought it would be interesting to run the same test on MarketingExperiments’ Best of the Month newsletter send, which is sent to readers on the first of every month. Similar to the Best of the Week email send, the Best of the Month features the most popular articles of the month, gives a recap of the latest Web clinic and provides a description of the upcoming Web clinic. This was done with the help of Ken Bowen, Managing Editor, MarketingExperiments.

The Best of the Month send has more information in it, so it is naturally a longer newsletter than the Best of the Week email previously tested. Therefore, it felt like a natural mental leap to assume that the shorter Treatment would once again get higher clickthrough rates as it seemed to be more reader-friendly.

As you can see with the below Control of the Best of the Month from August, there was a lot of copy in these sends.


For the Treatment, we once again cut the article summaries. In addition to losing this copy, we also cut the recap of the previous month’s research and instead just had the upcoming month’s Web clinic description. We ran this A/B split test for three months.


Due to the results of the Best of the Week testing, I knew what the results of this testing would be. If one audience responded positively the lessened copy, certainly so would another audience. The fact that we were virtually re-running the same test seemed to me to just be a formality.

Spoiler Alert: This is why testing is so imperative.

If I had applied the results from one test to all newsletter sends, I would have been doing our readers a disservice. Below are the results broken down by month.


Of the three months of testing, only one month validated. Although the Treatment had slightly higher clickthrough rates, it wasn’t significant enough to regularly validate as it did with the Best of the Week send. It appears that readers of the Best of the Month newsletter may find value in the summaries and extended copy that the Control version had.

Testing allowed us to change the MarketingSherpa Best of the Week send to better serve our readers and also learn that the same changes made to the Best of the Month send did not make a statistically significant difference to MarketingExperiments’ readers.



Testing is so important. I work for a company that values testing and is continually striving to best suit the needs of our readers. The editors and directors overseeing both MarketingSherpa and MarketingExperiments were very open to my experimenting with the format of a send that has been with the company much longer than I have.

And while the change of the format itself is important, so was the testing involved. It allowed me to really see how our readers were digesting the information that I was sending to them on a weekly or monthly basis. To better understand the patterns and behaviors of our audience only benefits me as a copy editor.

This meant that one audience preferred the lessened copy while the other may find value with more text . Without re-running the test, I would have blindly assumed that the MarketingExperiments audience had the same reading preferences as our MarketingSherpa audience.

Testing allows us to validate even that which we are sure to be true, and to humble us when this in fact turns out to be false.


You might also like

MarketingSherpa Summit 2016 — At the Bellagio in Las Vegas, February 22-24

A/B Testing: Ecommerce site’s 3,000 positive comments show why you can’t trust just one test

Testing and Optimization: Welcome send test results in 46% open rate for CNET

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: General Tags: ,

Andrea Johnson

Copywriting: 5 proven discoveries that strengthen copy

November 19th, 2015

Great copy isn’t about writing beautiful prose; it’s about knowing what to say to your prospects, when to say it and how to say it so they immediately become engaged, stay engaged and ultimately buy whatever it is you’re selling. Pretty words and design don’t matter as much as understanding what your prospects are thinking, what they expect during each stage of the buying process and then giving that to them.

That’s why MarketingExperiments has dozens of clinics focused on helping you write subject lines, headlines, body copy and more to help you achieve that. We call it “aligning copy with customer thought sequences.”

Get a condensed version of this information in the latest MarketingExperiments Web clinic. In about 20 minutes it distills more than 15 years of testing and research into five discoveries that can immediately help you write copy that sells. Watch it here.


Discovery #1: You have only seven seconds to arrest the attention of your prospect

That’s being generous. It’s critical to lead any copy with what the customer will value most about your product and nothing else. Show customers what’s in it for them immediately.

Version A leads with value: “Australia’s Most Trusted & Accredited Business Hosting Company.”


Version B doesn’t — it provides an explanation, but no value: “Business Dedicated Servers Australia.”


The Result: Version A achieved 188% more conversions.


Discovery #2: Never present the solution before building the problem

Never assume prospects know that they need what you’re selling; if they don’t know they have a reason to buy, they’re not going to.

Consider a company that sells a do-it-yourself solution. Unless you’re are a hardcore hobbyist, chances are you’re not going to know whether you need it. However, the Control in this test assumed you already know precisely what was wrong, and would be delighted that this product can rapidly and easily repair the problem. In contrast, the Treatment began with helping you diagnose whether you have the issue that the product resolves.

The Result: The Treatment achieved 36% more sales.


Discovery #3: Clarity trumps persuasion

Clearly communicate what’s valuable about your product upfront. If your prospect values it too, that will be more than enough to engage them. Only allow creativity to add to that value, and never allow it to detract. You may not win writing awards, but you’ll get far more sales being boring and clear than creative and confusing.

Consider this travel insurance provider. It started out with this vague, but clever, headline:

It switched to a headline that outlines value to the customer — peace of mind and the credibility of being in business for 35 years.

The Result: The Treatment achieved a 330% increase in conversions. Though other elements of the page were also changed in the Treatment, the clearer headline drove prospects further down the page and contributed to the lift in conversion.


Discover #4: Be ruthlessly unsentimental with your copy

To paraphrase Einstein: “If you can’t explain something simply, you don’t know it well enough.” Be sparse with words and, if you can’t, maybe you don’t know your product’s value well enough.

Generally, long copy creates friction, keeping people from moving forward in the sales cycle. However, in some cases where a product requires high commitment at a high cost, more copy has produced better results.

You just need enough copy to communicate the value of the product. Delete the rest. 


Discovery #5: Asks must align with expectations

If someone asked you to go on a first date and marry them in the same breath, you might think they were a little presumptuous at best. Yet, too many marketers do something similar in their campaigns. When they should be asking someone to “click here to find out more” they’re asking them to “buy now.” Or some iteration of that.

Consider this email that asks recipients to “Get Started Now.” That’s a pretty big commitment for someone to make based on an email.


This call-to-action was changed to “Estimate My Monthly Payment” which requires far less of commitment from the recipient.


The Result: Seven percent more people clicked through to get that estimate and 125% more people bought the product.


Use this information to immediately start increasing clickthrough and conversion by putting your copy to the test. Ask yourself:

  • Does my copy immediately arrest the attention of the prospect?
  • Am I sufficiently building the problem before presenting the solution?
  • Does my copy clearly express the value of my offer?
  • Am I using the proper amount of copy relative to the magnitude of my ask?
  • Does my call-to-action align with the expectations of my prospect?


You may also like

MarketingSherpa Summit 2016 — Get four days of proven insight to improve your campaigns fast

Email Marketing and Copywriting: Why you should send “spam”

Copywriting: Call-to-action testing and optimization

Copywriting: How to tip the scale so customers act

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: Copywriting Tags: , , , , ,

Daniel Burstein

Ecommerce: 6 takeaways from a 42-day analysis of major retailers’ holiday marketing

November 16th, 2015

In today’s MarketingExperiments blog post, we’ll look at the 2014 holiday campaigns of Amazon, Best Buy, Apple and Walmart and see what you can learn from their efforts. If you sell anything at all — from books to electronics to glow-in-the-dark toilet paper — you compete with at least one, if not all, of these retailers.


More than 500 data points gathered and analyzed over 42 days

To understand the information you’re about to see, let’s take a quick look at the methodology. 16 Web Research Analysts from MECLABS Institute (MarketingExperiments’ parent research organization) spent 42 days during the Black Friday (November 17 – November 29, 2014), Holidays (December 7 – 29) and New Year (December 30 – January 4, 2015) sales periods, analyzing three to five desktop webpages per retailer. This effort was led by Jonathon Yates, Market Intelligence Manager; Sarah Russell, Research Manager; and Gaby Paez, Associate Director of Research.

They assessed the campaigns using the MECLABS Value Proposition heuristic, asking:

  • How appealing (relevant, important and urgent) is the offer?
  • How exclusive is the offer?
  • How clear is the value of the offer?
  • How credible is the offer?   


Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: Marketing Insights Tags: , , , ,

Kayla Cobb

How Variance Testing Increased Conversion 45% for Extra Space Storage

November 12th, 2015

When it comes to testing, A/B testing typically steals the spotlight, casting its sister procedure, variance testing, in the shadows. However, according to Emily Emmer, Senior Interactive Marketing Manager, Extra Space Storage, that’s a mistake.

At MarketingSherpa MarketingExperiments Web Optimization Summit 2014, Emily presented on how her team was able to utilize variance testing to transform Extra Space Storage’s Wild West testing culture into a wildly successful testing environment.

Before the team conducted variance testing, the company’s testing environment was structured like a free-for-all. There were few, if any, set rules in place, and, according to Emily, the person with the highest title and the loudest voice typically had their test implemented. All of this changed after the Extra Space Storage team ran some variance tests.

Variance testing measures two identical Web experiences to determine a site or page’s natural variability. This procedure generally constructs the rules for subsequent A/B tests to follow.

By focusing on variance testing and translating the results from this procedure into rules for A/B testing, Extra Space Storage achieved a 45% increase in conversion rate from the previous year. Watch the below excerpt to learn the results of the team’s test, the rules they developed and Emily’s advice on when to start variance testing and how to implement it.

  Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: General Tags: , , , ,

Paul Cheney

Your Marketing Should Make Your Prospects Say “AHA!,” Not Just “Yes”

November 9th, 2015

At MarketingExperiments, we throw around a term a lot when we create content.

That term is “aha.”

I think it’s an extremely helpful term for marketers to understand and employ in their marketing.

But before I explain why it’s helpful, I want to first identify a serious problem in the world of marketing.

Often, even here at MarketingExperiments with our micro-yes inverted funnel, we are simply content to get a “yes” from our potential customers. A “yes” generally means a sale, a lead or a click, depending on where your realm of responsibility lies.


But what we don’t always get when a customer says “yes” is the maximum intensity of that “yes.” This translates ultimately to less momentum through the customer lifecycle, which translates to a lower lifetime value of a customer, which translates to lower revenue in the long term.

If you’ve ever bought a product that you needed because it was the lesser of two evils, you know exactly what I mean.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: General Tags: ,

Kayla Cobb

Lead Nurturing: Content-focused strategy leads to 74% lift in leads for Precor

November 5th, 2015

One of the most useful tools in the marketer’s workshop is content marketing. If utilized well, this strategy can turn even casual site visitors into potential prospects. However, how do you ensure that the right content makes its way in front of the right audience? That was one of the challenges Stephen Bruner, Marketing Manager, Vertical Markets, Precor, faced when he and his team sought to overhaul Precor’s nurture strategy and email tools.

Precor is a manufacturer of fitness equipment and caters to a global audience. It’s also the second largest industry manufacturer in the United States. The brand is owned by the multi-million dollar company Amer Sports, which is based in Finland.

During his presentation at MarketingSherpa Email Summit 2015, Bruner presented on how he and his team combined the lead nurture cycle with the CRM (customer relationship management), implemented a marketing automation effort and better utilized content marketing to reach the right audience. According to Stephen, content is an essential tool for lead nurturing.

“So this is my favorite part, right? Which is once we have the delivery mechanism, we’ve got the dynamic layer, then it comes to how do we nurture them and what do we do?” he said.

Stephen broke the buyer’s funnel down into four parts and listed the content Precor implemented at each step:

  • Top of the funnel: Buyer’s guide and infographics
  • Middle: Whitepapers (research)
  • Low: Service level interaction
  • Client: Customer service information

According to Stephen, “We wanted to put the customer’s interest first and say, ‘Let’s not confuse the customer. Let’s make sure that … during this period of time where the customer is an opportunity within our CRM system that they can have a one-on-one conversation with Sales.’”

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • Digg

Categories: General Tags: