What can you learn about your customers from a test that produced a negative lift?
This was the central idea behind the opening keynote presentation, “The Internet as a Living Laboratory,” at MarketingSherpa Email Summit 2013 from Flint McGlaughlin, Managing Director, MECLABS.
In today’s MarketingExperiments blog post, we’re reporting live from Email Summit and will share one of the experiments featured in Flint’s session along with three important questions every marketer should ask themselves when a test results in a conversion decrease.
We hope you use these questions to view your negative lifts not as a failure, but as valuable learning tools about your customers that you would not have learned otherwise.
But first, let’s get some background on the experiment …
Research Notes:
Background: REGOnline is event management software that lets users create online registration forms and event websites to manage their events.
Goal: To increase the number of completed leads on homepage
Primary Research Question: Which page will achieve the greatest addressable lead rate?
Approach: A/B multifactor split test
Control:
The researchers hypothesized that in the treatment they could increase the appeal associated with the value proposition in the offer by focusing more on the product and its specific features and benefits.
Treatment #1:
So, in treatment #1, the research team changed the following:
- Headline was written to focus more on the product.
- Specific features and benefits are utilized to express the value.
- The page emphasizes “Free Access.”
- Ensured product-centric value was communicated in subsequent steps.
Results:
The treatment lost to the control by -24.5%.
Even with a treatment that reduced form fields and increased the clarity of the value proposition, the control still outperformed the treatment.
So what explains the decrease in conversion?
Marketers often consider a negative lift a failure, but Flint considers it an opportunity to gain customer insight if you look at the analytics and begin asking deeper questions about the results.
Question #1: Ask why?
According to Flint, the key principles every marketers need to understand first is why.
“Asking ‘how’ leads to information; asking ‘why’ leads to wisdom. Yet marketers are all too busy asking ‘how,'” Flint said.
Here are some of the key “Why” questions offered as examples:
- WHY did the treatment decrease conversion by 25%?
- WHY did more people say “yes” to the control?
Question #2: Are we too quick to make changes?
“We as marketers are rushing around so often that we don’t have time to think,” Flint said.
He suggested the road to underperformance is paved with the best of intentions by marketers who took action out of impulse rather than insight. Another key takeaway the session offered was marketers should ask themselves is what guides action being taken.
- Sometimes we need to slow down in order to go fast. Action is overrated; action should be grounded in contemplation. Admittedly contemplation without action is anemic, but then action without contemplation is dangerous.
Question #3: Are we the philosophers of our organization?
Flint declared that the insights gained from testing should also drive the actions of your sales force.
- Indeed, the marketer should be the philosopher of the organization for the vigorous action of sales needs to be grounded in the rigorous contemplation of marketing.
The challenge Flint explains is a charge for marketers to begin thinking much deeper about marketing.
“The most important thing I want you to understand is that value proposition is at the very heart of enterprise,” Flint said.
Related Resources:
Email Marketing: 7 resources for email optimization
Email Marketing Timing: When is the optimal time to send your next marketing email?
Mobile Optimization: How the Ritz-Carlton Destination Club increased its mobile email traffic 243%