MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

ROI is not a result of A/B testing. It is a side effect.

Too many marketers waste time and resources assuming that if they simply create an A/B test, or test different elements on a page, they’ll automatically see results.

It’s more likely that a marketer will start split testing, and in a matter of months, they will hit a wall where it’s no longer profitable to run said tests. Take, for example, this chart from MarketingSherpa’s 2011 Landing Page Optimization Benchmark Report:

Click to enlarge

Of the 2,673 marketers surveyed, only 52% of them finished a complete testing cycle.

A little more than one half actually had the time, resources and commitment to push through and complete the project.

And even if they did complete the project, I’d love to see the chart on the completion rate for the second project. There’s probably another sharp drop off in that one.

So the question is, with testing being touted all over the Internet as the silver bullet in achieving a tangible ROI, why aren’t we seeing it?

Surely, if the ROI was so tangible, we’d have no choice but to complete our optimization projects, right?

  Read more…

“There are no expert marketers; there are only experienced marketers and expert testers.” – Dr. Flint McGlaughlin

Once upon a time, marketers could claim they knew what marketing collateral would generate the highest response from the customer. Now, with the advent of online testing, it has become more challenging to “make definite assertions” about which treatment will perform better.

The best we can do is pose  a hypothesis.

Of course, it never hurts when your hypothesis turns out to be absolutely right.

So to give you a 50/50 chance at gloating (even over a lucky guess), you can hypothesize which call-to-action performed better in the slides below.

Once you’ve studied the slides, go ahead a leave your hypothesis for which call-to-action won (and why) in the comments.

The commenter with the best hypothesis will get the recognition of his or her peers and be featured in the body of this post with a link to their site.

Here is the test we will highlight in today’s 4:00 p.m. (EST) Web clinic: Minor Changes, Major Lifts: How headline and call-to-action optimization increased conversion 45%.

Read more…

It’s happening in companies across the world as we speak … the highest paid person’s opinion (HiPPO) wins the day in the marketing department.

But does the person with the higher title and the higher salary always understand which headline (or PPC ad or landing page copy) will convert the highest?

Take a look at a recent test from our own labs.

Read more…

Early afternoon on day 3 of MarketingSherpa Email Summit 2012 in Las Vegas, I caught a case study with Justin Bridegan, Senior Marketing Manager, MECLABS, presenting with Frank Cartwright, SVP of Product and Platform Development, GamersFirst, a free online gaming website that offers premium subscriptions, items and packages for purchase.

The challenge at GamersFirst was getting more people through their sales funnel and turning them into purchasing customers.

To improve the performance of its sales funnel, GamersFirst extensively tested its email marketing.

Frank says, “We wanted to transform the email marketing division from a cost center to a profit center.”

GamersFirst’s funnel includes 100% of website users are registered, the company gets 80% of those registrants verified, 60% login to the site, and only 10% actually make a purchase.

One group of tests involved sending the validation reminder email to registrants. GamersFirst tested the format of the email – text or graphic, and the timing of the email – 24 versus 72 hours.

  Read more…

I’ve got a research question. Now what do I do with it?

A few weeks ago, Daniel Burstein wrote a blog about writing research questions. In that blog post, we emphasized the importance of asking “which” rather than “what” questions because a “which” question is clearly testable.

You might ask, “Which page format results in the most lead submissions?” or “Which price point generates the most revenue?” Both questions are clearly stated and include two key pieces of information:

• An independent variable you are going to test
• The dependent variable you will use to measure your results

To know if something is better, first you must know if it is different

With the research question on paper, we can easily create a hypothesis. For the former question: “All page formats will result in the same number of lead submissions.” This type of hypothesis is so famous in research circles that it has a name: “The Null Hypothesis.”

In general terms, the null hypothesis states that varying the independent variable will result in no change to the dependent variable.

In other words, you’re testing to see if changing the page (the independent variable) will change the number of leads (the dependent variable). After all, if there is no change, one cannot be any better than the other.

Why not “The new layout will result in the most lead submissions,” you ask. Because there is no concrete reason to know that there will be a change. Besides, if you already knew the effect of A on B, why would you need to test it?

  Read more…

What do you get when you divide Jacksonville Beach, Fla. by Arden Hills, MN? I’m sure there’s a punch line in there somewhere. However, if you were tracking your customers’ ZIP codes in a database you would have 32250/55112, or 0.585.

Never mind that it doesn’t make any sense to you and me to divide one ZIP code by another, but a statistical software package is happy to do exactly that for us. Most software just isn’t smart enough to realize that each ZIP code holds a discrete meaning from the next. It sees them as numbers: values which can be sorted in order and used in any type of calculation.

That is why researchers and statistical software packages classify variables into four main types: Nominal, Ordinal, Interval and Ratio.

In this post, I’m going to describe each type of variable to help you understand how they should be used, let you know how this can help improve your data collection … and, while we’re at it, help you sound sharp the next time you’re chatting with your data analyst at the water cooler.

- Read more…