Posts Tagged ‘a/b testing’

Web Optimization: How to get your customers to say heck yes!

April 7th, 2014 No comments

For e-commerce marketers, and many marketers with a subscription-based business, the value of the products they sell on the Internet is intangible when the purchase decision is made.

So who better to gain some conversion optimization advice from than an A/B tester who specializes in nonprofit marketing, the industry that must communicate the most intangible value of all – goodwill.

We brought Tim Kachuriak, Founder and Chief Innovation & Optimization Officer, Next After, into the studio and discussed:

  • The power of the value proposition
  • Creating a scarce resource
  • Commitment building
  • The value proposition train

I’ve known Tim for several years through his attendance at MarketingSherpa Summits, and am glad to have him as a featured speaker at the upcoming Web Optimization Summit in New York City. In fact, his Web Optimization Summit session was one of the things we worked on while he was in Jacksonville, Fla.


Below is a full transcript of our interview if you would prefer to read instead of watch or listen.

Read more…

Email Marketing: How A/B testing raised $500 million for Obama for America

March 10th, 2014 2 comments

At Optimization Summit 2013, attendees had a chance to gain a behind-the-scenes look at the one of the biggest marketing efforts ever. The result of this effort alone generated $500 million in donations, just from the email campaign.

The best part about this Summit presentation? All of the key takeaways can be applied to your own marketing efforts.

The 2012 presidential election was the chance for the team at Obama for America to apply A/B testing to email campaigns and learn what motivated their audience to donate.

Toby Fallsgraff, Email Director, and Amelia Showalter, Director of Digital Analytics, both of Obama for America, walked the Summit audience through their process in designing and executing email tests for President Barack Obama’s re-election campaign.

In this video excerpt from the full session, Toby and Amelia discuss the first lesson they learned from their work for Obama for America: content matters.


“We found this pretty cut and dry that no matter what we did, the different versions we would test and the subject lines actually made a difference,” Toby said.

The impact on testing the language and style of email content and subject lines was dramatic. According to Toby, the difference between sending the best email and the worst was huge – nearly $2 million in revenue.

One of the most consistent discoveries the team made was that the top-earning sends were personal and resonated with the audience in an approachable way.

“I think it gave the impression that there were real people writing these emails, that it wasn’t focus-grouped to death. It was something more off the cuff,” Amelia said.

To learn more about the team’s efforts, check out the full replay, which reveals how you can foster a culture of testing in your organization just like Obama for America did, and much more.

Also, if you’re interested, MarketingSherpa and MarketingExperiments will be teaming up again this spring in New York City to present Web Optimization Summit 2014.

Read more…

Radical Redesigns: Lifts vs. building customer theory

“Radical redesign” is a term we use often at MECLABS.

It’s used to describe treatments that are “radically” different from the control. We aren’t talking about changing some button copy from “Buy” to “Buy Now.” We’re talking new themes, layouts, copy and even functionality on the page.

Radical redesigns contain many variables making it difficult to isolate specific elements contributing to the results of a test.

Today’s MarketingExperiments Blog post will focus on several of the pros and cons of radical redesigns, but I’d first like to provide you with a little more context on testing in this fashion and how it may impact future testing.

Lifts are awesome.

That means your treatment(s) had a positive effect on your key performance indicator (KPI). You may have increased clickthrough rate, conversions or conversion rates, or possibly even leads.

Your treatment won, and you also learned how the variables you changed in the treatments resulted in an optimized page.

However, identifying specifically what, how and why these changes made an impact is what makes building your customer theory even better. Discoveries are important to customer theory because they can be attained regardless of whether your treatments “win” or “lose.”

Discoveries are also valuable in informing future testing.

Let’s say you changed some button copy. If your treatment wins and you see a lift, you learn that the button copy you changed resonated with the visitor.

But if the treatment loses, you’ve discovered that it didn’t entice the user to click, and you need to continue testing to determine the optimal copy to increase visitor engagement with the button.

Either way, you’ve gained valuable insight about your customers with your test.

However, this isn’t always the case with radical redesigns.


Radical redesigns may provide a lift, but they also leave a lot of unanswered questions 

There’s no doubt about it, radical redesigns are fun. You are able to play with designs. You can try all of your cool ideas. You can make landing pages or email treatments that look so much prettier (or uglier) than the control version.

Radical redesigns are often beneficial and can be a quick way to optimize your page if you are seeing multiple opportunities to add value and decrease friction and anxiety.

When radical redesigns win, it’s great. It means visitors loved your new designs, copy or functionality. Your hypothesis was correct, and the new page increased conversions. You achieved a lift, and you learned that whatever you did to the page resonated well with the visitor.

But you might have to ask yourself, “What else did I learn?”

You just don’t know what you did to the radically redesigned page that made it any better and that insight may be lost as to where to test next.

Was it the new headline? The altered layout? The aesthetics you added? Or was it that updated functionality?

Often times, the little voice that wants to know what you learned is silenced by the increase in conversions.

When radical redesigns lose, however, that’s a different story. While they are fun to plan and test, the sad fact is that when they lose, you are often left back at square one.

There is not much you can learn from an underperforming radical redesign and here’s why:

  1. You didn’t see a lift. You didn’t devise a new treatment that boosted conversions to implement. It lost. The only lesson you learned was that the control was better, which just leaves you back where you started.
  2. You don’t achieve as many valuable discoveries about your customers. What aspect of the page didn’t resonate with visitors? Maybe they liked the new layout, but the headline turned them off. You’ll never know. Therein lies the risk with radical redesigns.

Keep in mind, this post is not meant to deter you from radical redesigns. As stated before, radical redesigns are a great way to make many positive changes to a page when you have diagnosed specific shortcomings.

The idea here is make you aware of some of the pitfalls of testing radical redesigns.

Read more…

Copywriting: Do you take your prospects on a journey?

February 27th, 2014 3 comments

You’ve seen the statistics. Customers receive 12 million billion marketing messages a day.

Plus they’re busy, and have short attention spans.

So you may think, “I have to get my sales message and value prop to my customers as quickly as possible.”

But your goal as a marketer is not to get quick information in the hands of a customer. It’s to take them on …


The buyer’s journey

Let’s use “Star Wars” as an analogy.

George Lucas could have made a two-minute video on YouTube and said, “So … they’re brother and sister. And on top of it, the dude he’s fighting is actually his dad. Weird, huh?”

But if he did, I’m betting he wouldn’t have this level of brand loyalty more than 30 years later.

Storytelling is powerful.

It helps people see a new way of looking at the world. As a marketer, that includes how the world would be with your product or service in it.

By taking your prospects through a story, you help to welcome them into the world of your product, help them drop their defenses to actually hear what you’re saying, and get them to internalize your value proposition.

Your challenge is to decide how every element of your marketing can take them on that journey. For a simple purchase, this journey may happen in a single email or print ad. For a considered purchase, it may occur across an email drip campaign, nurture track or an entire marketing funnel.

You can watch the free MarketingExperiments Web clinic replay, “Copywriting on Tight Deadlines: How ordinary marketers are achieving 103% gains with a step-by-step framework,” to learn more about how story connects to the conversion process.


Photo attribution: Star Wars Blog

Read more…

Conversion Rate Optimization: Why is split testing so powerful?

February 13th, 2014 4 comments

Imagine if you could split test your life.

Essentially, you would never be faced with a conundrum again. You could choose to both send your daughter to the state school and the Ivy League college.

You could marry your high school sweetheart and that special someone you just met.

You could buy the Prius and the Tesla.

Then, you wait and you measure.

You would finally determine empirically which decision is best.

Of course, you can’t A/B test your life. That’s why there are 7,139 books in Amazon’s decision-making and problem solving category.


Don’t guess about your customers – know

But you, as a marketer, can split test. If you don’t know which subject line or headline will work, try them both and let the customer decide.

You can use Web optimization to make the best decisions:

  • Make some assumptions based on what you’ve learned about your customers
  • Use those assumptions to create treatments that test different hypotheses about the customer
  • Split your audience with an equal amount of similar customers being exposed to each treatment
  • Review and validate the results
  • Determine what can be learned about the customer and use that knowledge to repeat the process, becoming a little wiser each time

This process can add “thousands and thousands of dollars” in monthly revenue, as it did for

Web optimization can increase referrals and membership renewals, as it did for AARP.


What have you been able to do with Web optimization? Let us shine a light on your efforts.

Perhaps you’ve already discovered the power of split testing.

Of learning instead of guessing. Of experimenting instead of assuming.

You’ve discovered that experimenting and learning easily replace assumptions and guesswork about your customers.

If so, we’d like to hear your story even if the rest of your company has yet to catch up.

Others overlooking the power of this evidence-based technique is what frustrates many successful optimizers.

The good news is there’s a place where marketers can come together and learn what works in website optimization from each other. That place is Web Optimization Summit.


This year’s Summit will be in New York City in May. If you have a story, we’d love to hear it. I encourage you to fill out the speaker application and share what you’ve learned with other marketers.


You may also like

Customer Theory: What do you blame when prospects do not buy? [More from the blogs]

Customer Theory: How we learned from a previous test to drive a 40% increase in CTR [More from the blogs]

The Web as a Living Laboratory: The Three Most Important Discoveries from Over a Decade of Experimentation [Video]

Customer Theory: What do you blame when prospects do not buy?

February 10th, 2014 No comments

The effort and money that you’re investing in your marketing is predicated on one thing – that you understand your customer.

What good is a print ad, an email or a marketing automation investment if it doesn’t deliver a message that alleviates a customer pain point or helps a customer achieve a goal? They won’t act if the message doesn’t hit them square between the eyes.

Let me give you an example of faulty customer theory. Uber, a mobile car hailing service, is coming to Jacksonville. I recently received a push poll phone call clearly supported by the frightened taxi industry.

The main message seemed to be that Uber is cheaper because it uses unregulated (and, therefore, unsafe) drivers.


How often are you delighted by cab drivers?

What struck me was how far off their customer theory was from my actual wants and needs. I, for example, chose to take the BART from the airport to the hotel for Lead Gen Summit 2013 – not because it was cheaper (MECLABS was paying the bill either way, so it was free for me), but because riding in a cab is a miserable experience.

Plus, I’m putting my life in the hands of someone who will cut across three lanes of rush hour traffic with no turn signal to drop a passenger off 45 seconds quicker. Goodbye, safety argument.

The reason Uber, Lyft and other car hailing mobile apps are gaining traction is because they’ve found a way to create a better customer experience. Think about it. When was the last time you were delighted by a cab ride? In fairness, there was one time for me in Los Angeles. A kind driver gave me a quick tour of Bel Air during what limited free time I had on a business trip.

Here’s why the taxi industry struggles to realize the true threat.


We will tend to blame external rather than internal reasons when customers don’t buy

You put in so much time marketing your company and your clients that it becomes difficult to see the flaws customers see with unbiased eyes.

This is why A/B testing can be so valuable.

Actually forming hypotheses, testing these hypotheses in real situations with real customers, and then building a customer theory over time that informs everyone in your company about what customers really want is essential.

When you have your customer theory right, marketing can focus on clearly communicating how it can fulfill customers’ needs and wants.


Discover what customers want

Of course, A/B testing is only one way to gain customer intelligence. So to gain a perspective beyond my own, I asked Lindsay Bayuk, Senior Product Marketing Manager, Infusionsoft, for her perspective.

“Understanding what your customers want starts with understanding the problem they are trying to solve. First, define who your best customers are and then ask them about their challenges. Next, ask them why they love you,” Lindsay said.

Lindsay said some great ways to collect both quantitative and qualitative data on your target customers include:

  • Surveys
  • Interviews
  • A/B tests (see, I was telling you)
  • Sales calls
  • Feedback loops

The email registration process is another opportunity for learning more about your customers.

Ali Swerdlow, Vice President, Channel Sales & Marketing, LeadSpend, added, “Preference centers are a great way to gather data about your customers. Then, use that data to segment your list and message different groups accordingly.”

Read more…