Daniel Burstein

Conversion Rate Optimization: Why is split testing so powerful?

Imagine if you could split test your life.

Essentially, you would never be faced with a conundrum again. You could choose to both send your daughter to the state school and the Ivy League college.

You could marry your high school sweetheart and that special someone you just met.

You could buy the Prius and the Tesla.

Then, you wait and you measure.

You would finally determine empirically which decision is best.

Of course, you can’t A/B test your life. That’s why there are 7,139 books in Amazon’s decision-making and problem solving category.

 

Don’t guess about your customers – know

But you, as a marketer, can split test. If you don’t know which subject line or headline will work, try them both and let the customer decide.

You can use Web optimization to make the best decisions:

  • Make some assumptions based on what you’ve learned about your customers
  • Use those assumptions to create treatments that test different hypotheses about the customer
  • Split your audience with an equal amount of similar customers being exposed to each treatment
  • Review and validate the results
  • Determine what can be learned about the customer and use that knowledge to repeat the process, becoming a little wiser each time

This process can add “thousands and thousands of dollars” in monthly revenue, as it did for Cars.com.

Web optimization can increase referrals and membership renewals, as it did for AARP.

 

What have you been able to do with Web optimization? Let us shine a light on your efforts.

Perhaps you’ve already discovered the power of split testing.

Of learning instead of guessing. Of experimenting instead of assuming.

You’ve discovered that experimenting and learning easily replace assumptions and guesswork about your customers.

If so, we’d like to hear your story even if the rest of your company has yet to catch up.

Others overlooking the power of this evidence-based technique is what frustrates many successful optimizers.

The good news is there’s a place where marketers can come together and learn what works in website optimization from each other. That place is Web Optimization Summit.

 

This year’s Summit will be in New York City in May. If you have a story, we’d love to hear it. I encourage you to fill out the speaker application and share what you’ve learned with other marketers.

 

You may also like

Customer Theory: What do you blame when prospects do not buy? [More from the blogs]

Customer Theory: How we learned from a previous test to drive a 40% increase in CTR [More from the blogs]

The Web as a Living Laboratory: The Three Most Important Discoveries from Over a Decade of Experimentation [Video]

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Analytics & Testing Tags: , , , ,



  1. Nikki
    February 13th, 2014 at 21:30 | #1

    This is good information. What is a good sample size to use when using a split test?

    • February 24th, 2014 at 09:55 | #2

      Thanks for your questions Nikki and I commend you for even asking it. Many marketers overlook sample size.

      I cannot provide one perfect number for the right sample size (e.g., 10,000) because there are many factors to consider. This blog post has links to further resources about sample size sufficiency — A/B Testing: Split tests are meaningless without the proper sample size — I hope you find it helpful. And let us know if you have any other questions.

  2. February 14th, 2014 at 12:34 | #3

    This is great advice. What are your recommendations for how to conduct split testing?

  1. No trackbacks yet.