Andrea Johnson

4 Threats that Make Email Testing Dangerous and How a Major Retailer Overcame Them

October 2nd, 2014

To test emails, you just send out two versions of the same email. The one with the most opens is the best one, right?

Wrong.

“There are way too many validity threats that can affect outcomes,” explained Matthew Hertzman, Senior Research Manager, MECLABS.

A validity threat is anything that can cause researchers to draw a wrong conclusion. Conducting marketing tests without taking them into account can easily result in costly marketing mistakes.

In fact, it’s far more dangerous than not testing at all.

“Those who neglect to test know the risk they’re taking and market their changes cautiously and with healthy trepidation,” explains Flint McGlaughlin, Managing Director and CEO, MECLABS, in his Online Testing Course. “Those who conduct invalid tests are blind to the risk they take and make their changes boldly and with an unhealthy sense of confidence.”

These are the validity threats that are most likely to impact marketing tests:

  • Instrumentation effects — The effect on a test variable caused by an external variable, which is associated with a change in the measurement instrument. In essence, how your software platform can skew results.
    • An example: 10,000 emails don’t get delivered because of a server malfunction.
  • History effects — The effect on a test variable made by an extraneous variable associated with the passing of time. In essence, how an event can affect tests outcomes.
    • An example: There’s unexpected publicity around the product at the exact time you’re running the test.
  • Selection effects — An effect on a test variable by extraneous variables associated with the different types of subjects not being evenly distributed between treatments. In essence, there’s a fresh source of traffic that skews results.
    • An example: Another division runs a pay-per-click ad that directs traffic to your email’s landing page at the same time you’re running your test.
  • Sampling distortion effects — Failure to collect a sufficient sample size. Not enough people have participated in the test to provide a valid result. In essence, the more data you collect, the better.
    • An example: Determining that a test is valid based on 100 responses when you have a list with 100,000 contacts.
Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Analytics & Testing Tags: , , , , ,

Jon Powell

Why Subtle Changes in Button Copy Can Significantly Influence Clicks

September 29th, 2014

Earlier this year, a team of analysts approached me about a closed case that was reopened for additional interpretation. It was about button copy, and the results were initially baffling.

Here are three of the prominent treatments and their respective clickthrough rates, each with a statistically significant difference in comparison:

 

Does this look familiar? While Daniel Burstein did a great job covering the “what” of this test earlier this year in this MarketingExperiments Blog post, he did not have opportunity to go into the why:

  • Why didn’t “Start Free Trial” win?
  • Isn’t there more value by highlighting the word “Free?”
  • Why is it that the word “Now” in “Get Started” was the difference between underperforming or outperforming “Start Free Trial?”

Everyone in digital marketing is convinced that a call-to-action is a button or a link — something that people can click, or touch, and it will take them off the current view and into another. Because of this preconception, they often create and improve their calls-to-action with the same kind of tunneled focus.

Unfortunately, that’s not how the customer looks at it.

 

To the customer, the clickable thing has no meaning outside of its surrounding context

Take this classic example of context from Leonard Mlodinow’s book Subliminal.

 

Read this sentence:

“The cooking teacher said the children made good snacks.”

 

Now read this one:

“The cannibal said the children made good snacks.”

 

The meaning of the word “made” has significantly changed hasn’t it? In fact, the meaning of that one word is dependant on the context in which it is placed.1

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Copywriting Tags: , , , , , ,

Selena Blue

Website Optimization: 6 tips for effective 404 pages

September 25th, 2014

I’ve come across some beautifully designed 404 pages over the years. However, once the one second of artistic appreciation ends, I’ve been left confused and lost. The designers of those pages, whether Web designers or marketers, missed a great opportunity.

Your 404 page should have two objectives:

  1. Notify visitors they’ve encountered a problem of some sort while landing on the page they wanted
  2. Guide the visitor to what they wanted or to something else of value

A 404 page doesn’t have to be a dead end, or even a “Go to [Homepage]. Do not pass Go. Do not collect $200” card. It can be a user-friendly and functional page. It can have a greater purpose.

Read on to learn six tips to creating more effective 404 pages. You’ll also see “Not this, But this” examples demonstrating the tips.

Consider this blog post a creative swipe file of 404 pages, if you will.

 

Reduce Friction and Anxiety

 

Tip #1. Establish where visitors have landed

Not every visitor on your website who lands on a 404 page will have come from somewhere else on your site. When another site links back to your website incorrectly, or with an expired link, you potentially have visitors who are brand new to your site.

If your 404 page provides no way for new visitors to know where they are, chances are they’re going to press the back button never to be seen again. You just lost an opportunity for a new customer or reader. On the same note, if you provide no useful link for them, the back button is where they’re probably going to go.



The “Not this” page gives me nothing. Am I on a farm page? A livestock for sale website? A personal site for someone who really loves pigs? I have no clue based on the webpage.The “But this” example keeps its logo in place so visitors immediately know where they are. The copy of the page also gives clues as to where they are and what they can do on the site, even on the 404 page. 

 

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Site Design Tags: , , , , ,

Hannah Morrell

Email Optimization: Testing best time of day and day of week for email interaction

September 22nd, 2014

When do you check your personal email? Do you let it build up throughout the work week and go through it during the weekends? Do you check it on Monday when you’re also sorting through your work email? Or do you check it while you’re at lunch or on a quick, but much-needed, break from work?

In today’s MarketingExperiments Blog post, we’re going to explore which times of the day and days of the week people are most likely to interact with their emails — two questions of optimal interest for any emailing campaign.

 

Testing  the time of day when people interact with email

In email testing, we focus so much on the content and landing page of the email, but that hard work won’t pay off if email recipients don’t open or clickthrough the email. We wanted to get a better understanding of when people interact with emails to determine the best time of the day and day of the week to send promotional emails.

First, we began testing what time of day people are most likely to open and interact with emails.

Emails were currently being sent out on Mondays and Wednesdays at 7 a.m. EST. We hypothesized that by sending emails at various times throughout the day, we would learn the optimal times recipients are most likely to open and clickthrough their emails.

In an A/B split test, we sent a promotional email at 7 a.m., 3 a.m., 3 p.m. and 7 p.m. EST on a Monday. We wanted to isolate the general times of day people may be interacting with their email.

3 a.m. was tested to determine if people were more likely to interact with their emails as soon as they wake up in the morning and before they start their day, while 3 p.m. would tell us if people were checking their emails in the afternoons.

Lastly, 7 p.m. results would show that recipients were more likely to check and interact with their email in the evenings or later at night.

By sending emails at 7 p.m. EST instead of 7 a.m. EST, we saw a 12% lift in open rate:

  

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Email Marketing Tags: , , , , ,

John Tackett

Search Marketing: How a simple copy change increased conversion 21%

September 18th, 2014

Serving customers effectively starts with intelligence.

It’s not the kind of intelligence needed to solve Sudoku puzzles or carry home a victory on trivia night, but rather it’s what you really know about your customers:

  • What keeps them up at night?
  • How could your product or service transform their careers?
  • How could what you’re offering transform their businesses?

Yes, to serve your customers effectively, you have to understand how your products or services are relevant enough to effectively relate to their needs.

It’s also worth mentioning that PPC testing can help you build your customer theory, often on the cheap.

In today’s MarketingExperiments Blog post, let’s look at some recent PPC ad experiments that show how you can better use testing and optimization to help you understand your customers’ needs and ultimately build a deeper connection with your customers.

But first, here’s quick overview on the test background:

Background: A CRM software solution for small and large businesses.

Goal: To increase the total numbers of clicks.

Primary Research Question: Which PPC will generate the most clicks?

Approach: A/B multifactorial variable cluster

 

Control 

Control

 

The original ad emphasizes the fact that the software is award-winning and can be fully integrated into a business.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Paid Search Marketing (PPC) Tags: , , , , , ,

John Nye

Ecommerce: 2 tips I learned from a garage sale

September 15th, 2014

My father passed away unexpectedly five months ago. As if that wasn’t enough of a tragedy, the situation left my mother and me in a position we never imagined being in — she could lose her home. Quickly, I set up a GoFundMe page to prevent this.

Despite adversities and setbacks, my mother has a positive outlook and is moving forward. She decided to host a garage sale to help cover some costs. The first sale she held earned enough income to cover some costs and inspire hope.

You may ask, “Why is this guy starting off his MarketingExperiments Blog post with a personal story? What can readers learn about marketing from a life event?”

In helping with my mother’s second garage sale, I gained two key insights that I’ve been able to use as a MECLABS research analyst.

 

Prominence and Eye-path: A match made at checkout

Most of us are aware that prominence is crucial to the discovery of any product on any page.

This cannot be truer than when it came to a convection oven we sold at the garage sale. It was a relatively high-priced, chunky item that had been used twice. We knew it would not be an easy sell.

So we prominently displayed the appliance on one of the very first tables in front of our enclosure – front and center in the sale and near our checkout. It was within our customers’ eye-path as they browsed and made small talk.

Our magic worked when a customer noticed the item when he began speaking to us. It was one of the first items sold.

This lesson can be directly applied to your website. Whether it’s a beefed-up kitchen appliance in a garage or a newly released product on your website, the product needs to be easily found for it to convert.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Ecommerce Tags: , , , , ,