Archive

Posts Tagged ‘email testing’

Copywriting: Brevity is the soul of marketing

November 20th, 2014 1 comment

I’ve always loved this quote:

“Brevity is the soul of wit.” – William Shakespeare

To me, its beauty rests in the powerful meaning packed in six simple words. Brevity can also be used as a tool to aid your marketing, as I discovered from a recent email experiment.

But first, a little more detail about the experiment.

Background: A global producer of high-quality audio equipment and accessories.

Goal: To increase clickthrough rates in an email.

Research Question: Which email will generate the highest clickthrough rate?

Test Design: A/B multifactor, radical redesign split test

 

Control email-test-control

 

In a preliminary review of the control, the MECLABS research team hypothesized the control was at risk of underperforming and could use some strategic tweaks.

Read more…

4 Threats that Make Email Testing Dangerous and How a Major Retailer Overcame Them

October 2nd, 2014 No comments

To test emails, you just send out two versions of the same email. The one with the most opens is the best one, right?

Wrong.

“There are way too many validity threats that can affect outcomes,” explained Matthew Hertzman, Senior Research Manager, MECLABS.

A validity threat is anything that can cause researchers to draw a wrong conclusion. Conducting marketing tests without taking them into account can easily result in costly marketing mistakes.

In fact, it’s far more dangerous than not testing at all.

“Those who neglect to test know the risk they’re taking and market their changes cautiously and with healthy trepidation,” explains Flint McGlaughlin, Managing Director and CEO, MECLABS, in his Online Testing Course. “Those who conduct invalid tests are blind to the risk they take and make their changes boldly and with an unhealthy sense of confidence.”

These are the validity threats that are most likely to impact marketing tests:

  • Instrumentation effects — The effect on a test variable caused by an external variable, which is associated with a change in the measurement instrument. In essence, how your software platform can skew results.
    • An example: 10,000 emails don’t get delivered because of a server malfunction.
  • History effects — The effect on a test variable made by an extraneous variable associated with the passing of time. In essence, how an event can affect tests outcomes.
    • An example: There’s unexpected publicity around the product at the exact time you’re running the test.
  • Selection effects — An effect on a test variable by extraneous variables associated with the different types of subjects not being evenly distributed between treatments. In essence, there’s a fresh source of traffic that skews results.
    • An example: Another division runs a pay-per-click ad that directs traffic to your email’s landing page at the same time you’re running your test.
  • Sampling distortion effects — Failure to collect a sufficient sample size. Not enough people have participated in the test to provide a valid result. In essence, the more data you collect, the better.
    • An example: Determining that a test is valid based on 100 responses when you have a list with 100,000 contacts.

Ecommerce: 3 landing page elements to help increase product emphasis

July 14th, 2014 No comments

The elements on a product page are often one of the most underutilized tools a marketer has at their disposal. I say this, because let’s be honest, I’d wager few folks think of design elements on a product page in a “tool mindset.”

But in some respects, that’s exactly what they are, and ultimately, that’s how you will determine the kind of customer experience you build in ecommerce.

In this MarketingExperiments Blog post, I wanted to share three elements you can tweak to help emphasize important products and maybe even increase your revenue along the way.

 

Element #1. Size 

product-page-elements

 

Here’s an excellent example of how resizing a product image can help you place emphasis on it.

In the control, there were three products on the right sidebar and they were all equally weighted – that is a problem.

Nothing really stood out, which made drawing a clear conclusion for customers a little difficult.

In the treatment, instead of having three separate products on the page, the marketers hypothesized that a single product with a dropdown selection for a computer operating system would increase conversion.

Their hypothesis was right – the results from the tests included a 24% increase in revenue.

 

Element #2. Color 

email-product-testing

 

Here is another example of using elements in an email that you should pay close attention to because products are not trapped on pages in storefronts.

That perception is far from reality.

According to the MarketingSherpa Ecommerce Benchmark Study (download a complimentary copy at that link), email is one of the biggest drivers of ecommerce traffic.

In the treatment, the number of products were reduced, and bright red copy was used as supporting emphasis. I’m not fluent in Italian, but in any language, that is a good thing.

As you can see, color emphasis and copy now drive this email. From the changes in the treatment, I can intuitively understand the desired outcome:

  • I can order something at a great price
  • I get something free (gratis) as a thank-you gift
  • It only takes three easy steps to order

The treatment delivered a 24% increase in revenue with the right changes needed to have a powerful impact.

Read more…

Lead Generation: Capturing more leads with clear value prop communication

October 3rd, 2013 2 comments

According to the MarketingSherpa 2012 Lead Generation Benchmark Report, 51% of marketers surveyed indicated the most effective platform for testing their value proposition was through email marketing campaigns.

This is no secret to savvy marketers. Austin McCraw, Senior Editorial Analyst, MECLABS, also discussed how to discover the essence of your value prop through email at Email Summit 2013.

Jon Ciampi, Vice President of Marketing, CRC Health, did just that and revealed his strategy at Lead Gen Summit 2013, happening right now in San Francisco.

In his session, “Lead Capture: How a healthcare company increased demand for services 300%,” Jon shared with the Summit audience how understanding customer motivations, driving traffic, and clearly communicating the value proposition all helped his company capture a higher quality of leads.

At CRC Health, Jon developed nine value propositions, and broke that list down into problem- and solution-focused messages. He combined the company’s in-house list with a purchased list consisting of psychiatrists and therapists who refer their patients to CRC Health. Then, the team crafted email subject lines reflecting the different value propositions to test where the customer was in regard to researching the problem, or looking for a solution.

Through testing, Jon discovered a 14.49% clickthrough rate in the top-performing subject line, and this was problem-focused messaging rather than solution-focused messaging. For CRC Health, the process of searching for a rehabilitation center is most likely a first-time experience for customers. Therefore, understanding that these prospects are looking for different options related to their problem, rather than immediately solving the issue, was extremely important to targeting their needs. 

 

“What we found is with rehab, everyone is focused on the problem. With our in-house list, patient-focused messages were more motivating and increased clickthrough rates,” Jon said.

Even though he made a breakthrough with testing value propositions through email, he did encounter the fact that one size does not fit all, particularly with his audience, and even more specifically with a purchased list.

For psychiatrists opening CRC Health sends, their top message for open and CTR was scientific-based. The subject lines and topics that most resonated with this segment were “improving addiction treatment with science and research,” “outdated addiction treatments fail patients,” and “CRC Health as the strongest clinical supervision in the nation.”

However, the audience that preferred more relationship-based messages was therapists. Messages like “Treatment fails when therapists & clients aren’t aligned,” and “Most rehabs can’t provide effective clinical supervision” were the top performers for this segment of CRC Health’s audience.

“Overall, self-serving messages performed far worse than patient-focused messages. Patient-oriented problem statements motivated them as well,” Jon said.

Through value prop testing with his audience via email messaging, Jon learned much more about his audience and their motivations.

As an exciting result of value proposition testing, he discovered a 3x to 4x increase in demand for services. According to Jon, when testing began, both inquiries and admissions increased.

“One of the top things I learned is to look at funnel. What are the motivations of your customers? … Also, understand their language. Different buyers with different perspectives will affect how your messages are interpreted,” Jon concluded.

  Read more…

Email Marketing: What you can learn from an 80% decrease in clickthrough rate

February 13th, 2013 No comments

On the MarketingExperiments blog, we often share tests we conduct with Research Partners. Today’s post was run on our own marketing campaign.

The team tested a promotional email for the MarketingSherpa 2012 Mobile Marketing Benchmark Report.

 

CONTROL

Subject Line: [Just Released] New Mobile Marketing Benchmark Report 

Click to enlarge

 

The control featured general copy about using mobile in your 2013 marketing strategy and what tactics are working for mobile.

After evaluating the control, the team hypothesized the email did not have information about the insights prospective customers will receive from reading this benchmark report.

From that analysis, the team crafted …

Read more…

Timing and Email Marketing: Sunday generated 23% higher clickthrough than Tuesday in test

January 25th, 2013 2 comments

In the MarketingSherpa Email Marketing Benchmark Report, 79% of marketers found delivering highly relevant content to be an either somewhat or very significant challenge.

When you break down email relevancy to its core components, it is essentially a combination of:

  • Getting the right message …
  • To the right person …
  • At the right time.

While we often write about email messaging on the MarketingExperiments blog, today we’ll focus on that last element of relevancy – timing – in the following experiment with a MECLABS Research Partner

 

EXPERIMENT

Background:  A large financial institution offering a financial service requiring an application to consumers

Goal:  To increase the number of completed applications

Primary Research Question:  Of the send times tested, which time will result in the highest rate of completed applications from delivered emails?

“We conducted some research on the best times to send email, and the test was intended to see when current customers were more likely to complete an action,” said Ashley Hanania, Senior Research Manager, MECLABS.

 

EXPERIMENT DESIGN

The test had a total of 14 treatment paths. Each path had the same subject line, email body copy and design.

The only treatment value that was tested was send time. Two emails were sent each day of the week, one at 3:00 a.m. EST and one at 3:00 p.m. EST. The recipients were all in U.S. time zones.

“We tested 3 a.m. EST because the email would be first in your inbox, regardless of where you lived,” Ashley said. “This was also taken into consideration for the 3 p.m. EST send, because every recipient would be in the same mindset, afternoon work/weekend activities, as opposed to a 6 p.m. EST send, where the East Coast would be making their commute back home and the West Coast would still be at work.”

Read more…