MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

When you look at some of the emerging topics in marketing – analytics, metrics, A/B testing, conversion optimization and the like – there seems to be a growing interest among marketers in what I like to call … evidence-based marketing.

AB Testing

Conversion Optimization

Marketing, perhaps more than other business disciplines, has tended to rely on the “golden gut,” or a few high-performers. But, as the technology to measure and test marketing campaigns has become increasingly cheaper and more available, the ability to make marketing decisions based on real-world performance is a reality.

To help marketers become more evidenced-based, we’ve crafted the agenda for Optimization Summit 2013 in Boston around three key phases. No matter what phase you’re at, you can improve your ability in that phase, and try to move towards a more mature evidence-based marketing program. Here is a brief explanation of each phase, along with a sampling of resources to help you prepare for Optimization Summit 2013, or simply improve your own efforts.

Optimization is determining the value of the company, each product and each offer, and improving the communication of that value to the customer.

So, this includes landing page optimization and conversion rate optimization. But, it also includes messaging, copywriting, design and presentation.

It breaks down to two areas of collecting information – internally and externally. What have you learned from previous efforts to can improve your marketing campaigns? What have you learned from outside research, such as case studies, benchmark reports and networking with your peers?

Here are a few examples of resources to help you in the Optimization phase:

Converting PPC Traffic: How strategic keyword placement increased conversion by 144%

Email Optimization: A single word change results in a 90% lift in sign-ups

Optimizing Shopping Carts for the Holidays: 6 last-minute changes you can make to your shopping carts to increase conversion

PPC Optimization: Tips from your peers on regional differences, Google Product Listing Ads, distracted visitors and offline conversion

You’re in the Optimization phase when you’re making a concerted effort to improve your marketing campaigns based on what you have learned and think will work. However, this is only the beginning.

Read more…

Online testing is an exciting venture. While it requires a number of up-front resources and a somewhat long-term commitment, it really can transform your company’s earned media online into a living laboratory for building customer theory.
I’ve learned a few critical lessons in my journey into online testing, and this one is likely the most important.

Tests never perform as expected

We believe as long as we get a test set up and live, that one version will be performing much better than the other, and everything will be easy to figure out. In fact, we don’t just expect this for the final test result, we expect it for the daily results, too, like this graph:

Example #1: Real daily test results from an archived test

In the graph above, Treatment 1B is mostly beating the control, both lines are generally going up and down together, and the results seem to show that Treatment 1B is likely more effective.

But, what do you do when you see results like in Example #2?

Example #2: Real daily test results from an archived test

The lines trend somewhat together, but crisscross.  One is winning, then the other, then back again. If one is showing a win in terms of results, can you really take it seriously after seeing this graph?

What happens if two different versions beat the control by virtually the same amount at the same time?

Example #3: Actual treatments from the same test that both significantly beat the control version by virtually the same amount of difference

I can’t recall exactly how many times this has occurred here in the laboratory, but it happens often. Tests don’t always perform the way we expect, or want them to.

You can’t rewind what’s already been played

Once you have your data and your test is closed, there’s no going back to try and get something that you need if it wasn’t already being collected.

That means if you don’t have a solid lift, chances are you won’t be able to truly distill the learning to make up for that investment of resources. Even if you have a lift, you’re not set up to understand why.

You’ve got to over-measure to truly understand

Anticipating this issue, our team set up a particular homepage test to track all the expected metrics PLUS additional channel-specific metrics (i.e., the difference in performance between referral source traffic and search-engine traffic). Keep in mind that for all three versions, we had to collect eight form fields per submission for it to be counted as a lead.

Control

Treatment #1

Treatment #2

Read more…

On a recent MarketingSherpa webinar, sponsored by Paramore, I discussed statistical analytics techniques with Benjamin Fillip, Data Analyst, MECLABS …

Ben chose the techniques to feature on “Four Techniques to Improve Analytics Based on Customer Knowledge” from his experience working with MECLABS Research Partners. These are the same four techniques the MECLABS team of data scientists typically uses at the beginning of a Research Partnership to help guide testing and optimization.

Be Al Roker, not Tom Brokaw

Recent research in the MarketingSherpa 2013 Marketing Analytics Benchmark Report indicates 48% of marketers are using analytics platforms to customize reports, but only 24% are creating and testing hypotheses.

Read more…

As we all know, the holiday season is upon us. What does that mean? Shopping! Black Friday, Cyber Monday, Green Monday, Purple Tuesday. … Yes, I made the last one up, but I digress.

This time of year is one of the biggest, and often the biggest, revenue-generating periods for most e-commerce and B2C companies. So one of the most frequent questions we hear around here at The Lab is “To test or not to test?” Deep, I know.

I’m an advocate for testing year round. However, you must be aware of external factors out of your control that may change your prospect’s behavior. While the holiday season may bring a large number of visitors to your site — which is great for testing — you don’t want to risk losing those valuable visitors to an underperforming test.

So what do you do? Well, in my opinion, two factors will make the decision for you.

Read more…

You’ve finally set up tracking on your site and have gathered weeks of information. You are now staring at your data saying, “Now what?”

Objectively interpreting your data can be extremely overwhelming and very difficult to do correctly … but it is essential.

The only thing worse than having no insights is having incorrect insights. The latter can be extremely costly to your business.

Use these six simple steps to help you effectively and correctly interpret your data.

Read more…

Some marketers simply drop two landing pages into a split test tool, click a button, and then push live the page “winner” with the larger results number.

If you really want to benefit from split testing, you need to do a little more. You need a basic understanding of what you’re really doing when you’re testing. That includes validity, which is, at its most basic level, an assurance that the results from your tests actually reflect what is going on in the real world.

Let me show you an experiment with a MECLABS Research Partner in which an understanding of validity helped the team find a conversion lift that would have otherwise been missed.

Background: Consumer company that offers online brokerage services

Goal: To increase the volume of accounts created online

Primary research question: Which page design will generate the highest rate of conversion?

Test Design: A/B/C/D multi-factor split test

CONTROL

Click to enlarge

Read more…