MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

This time of year, many marketers are beginning to execute on their new marketing plans. However, sometimes you have to deviate from the plan and just need a sale or lead generation lift… RIGHT NOW!

When your boss or client challenges you to gain a quick conversion increase on your landing pages, what tools do you turn to in your marketing toolbox?

In Wednesday’s Web clinic – Rapidly Maximizing Conversion: How one company quickly achieved a 53.9% lift with a radical redesign – MECLABS Managing Director Flint McGlaughlin will share our top discoveries around how to quickly improve your conversion rate.

But before we share what we learned, we wanted to hear from you. Here are a few of our favorite “quick hit” tips from your peers …

- Read more…

As I prepare to wade through hundreds of submissions for the MarketingSherpa 2012 Marketing Wisdom Report, (sponsored by HubSpot) I was compelled to take a final glance at the 2011 edition.

While combing through the pages, many of last year’s submissions evoked some forward-thinking thoughts for 2012. Here are just a few of the standouts…

- Read more…

The majority of B2B organizations are increasing their marketing budgets for inbound marketing tactics. One of the most popular of those inbound tactics is virtual events and webinars, with 60% of marketers increasing their investment according to the MarketingSherpa 2011 B2B Marketing Benchmark Report.

“It is essential for organizations to gain the trust of their buyers before they can hope to convert them,” said Jen Doyle, the Benchmark Report’s lead author. “Webinars offer an effective platform to improve thought leadership and reputation, both essential components to winning trust. The cost effectiveness of webinars is just the icing on the cake, so many organizations are shifting to include webinars as part of their marketing plans.”

Of course, a webinar isn’t very effective if no one attends. So in today’s MarketingExperiments blog post, Gaby Paez and I will give you some test ideas for those all important webinar invite emails (and if you’d like to see how we craft our own webinar invite emails, just sign up) by reviewing a live optimization submission from The Chronicle of Philanthropy.

Gaby is associate director of Research at MECLABS, and you can hear more of her test ideas in the Web clinic replay, Email Messaging: How overcoming 3 common errors increased clickthrough 104%, along with some of the audience’s optimization advice for this submission.

Here’s the submission (and you can view it online as well) …

Click to enlarge

BACKGROUND

Email – Invitation to a paid social media webinar, “Going Mobile: How Nonprofits Succeed,” which features a bonus opportunity to gain access to “an exclusive discussion group” and three speakers:

• Felicia Carr, Senior Director of Online Communications, National Parks Conservation Association
• Jason Wood, Director of Digital Communications Services, Salvation Army National Headquarters
• Cody Switzer, Assistant Web Editor, The Chronicle of Philanthropy

Audience – Nonprofit professionals in fund-raising, marketing, social media and development

Objective – To get registrants for a paid webinar

  Read more…

It’s the holiday time, so let me talk about a holiday. Passover, naturally (hey, if you want to succeed in marketing, don’t follow the crowds).

In the classic seder, there are The Four Sons. For this blog post, I’m going to focus on “the one who does not know how to ask a question” because I think that’s a perfect explanation of where many marketers are right now with their testing practices. For example, according to Jeff Rice’s just released 2012 Email Marketing Benchmark Report, he found that 85% of marketers don’t even know why they’re running every test they run!

-

Click to enlarge

-

Take a look at the second field in this chart. Only 15% of marketers routinely define the question, objective, and key metric when running a test. Why bother taking the time to set up a test if you don’t know what you’re looking for in the first place?

My guess is, that like the fourth son referenced above, they simply do not know how to ask. So in today’s blog post, I want to briefly discuss how to write a research question. And in this Thursday’s MarketingSherpa webinar – Negative Lifts: Turning a 25% loss into a 141% increase in conversion – Junior Editorial Analyst Paul Cheney and I will be discussing in more depth how you can learn about your customers from tests, along with Tina Hou, the director of product marketing for webinar sponsor TRUSTe.

-

How to create a clear research question

While all of the elements in Jeff’s chart are important for running a valuable test (i.e. truly learning what really works with your customers), the research question will play the biggest role in guiding your test design.

That said, the research question is just one part of step #11 (“Test Comps”) in the MarketingExperiments Optimization process. Before our analysts begin to design tests using the MECLABS Test Protocol (in which they define the research question) and begin the iterative testing process, they do everything from determining the page objective to submitting comps for peer review (you can see our full landing page optimization process in session 7 of our paid Landing Page Optimization Online Course.)

While I obviously can’t cover the entire process in this blog post, if I can help you write a true research question, I can set you on the path to learning about your customers from your tests. This is a complex process, but if I had to simplify it into three steps, I would say…

-

1. Start with asking “what” you want to know

Clearly you’re running a test for a reason. Write that question down on a piece of paper. Go ahead, do it, I’ll wait.

Now pass that piece of paper (or email) around. Are all of the key players aligned that this is, in fact, what are you trying to learn from your tests?

For example, you may want to know “What is the best price for product X?” This is the variable you will test.

-

2. Turn this into a question of “which”

Good start. Except the only problem is, to truly answer the above question, you would have to test an infinite number of prices. And I’m guessing your time, resources, traffic, and patience are not infinite.

So to narrow your focus, you want to ask a question of “which.” Not only will this force you to think about exactly how you’re designing your test, it helps you create a testing-optimization cycle to continually learn about your customers from your tests and improve your marketing performance.

A year from now, when you’re been promoted three times for driving such impressive results, and the new hotshot your direct report’s direct report hired sees a test that asks, “Which of these three price points – $1, $2, or $3 – is best for product X?” he will know exactly what you tested. And exactly what you learned about the customer.

-

3. Add in your KPI

So now you know exactly which prices (technically speaking, which values of your variable) you need to design a test around. The next question you need to ask is – how do I pick a winner?

You certainly don’t want to write the rules after the fact. “My favorite analogy for this is throwing a rock in the forest and saying, ‘look, I hit that tree,’” said Phillip Porter, Data Analyst, MECLABS. “If you aren’t aiming for something before you start, how do you know if you hit what you aimed for?”

What KPI (key performance indicator) will help you determine which value is the winner? To reformulate our example question, you would say “Which of these three price points – $1, $2, or $3 – will generate the most revenue for product X?”

Now everyone on your team (and everyone on your team a year from now) knows exactly how you define “best.” If you don’t think through and define the question beforehand, you might just try to come up with an answer based on whatever metrics you had on hand after the test is run. For example, choosing sales instead of revenue, and picking a winner that sells more product but generates less money in your pocket.

You might also not even have the chance to redefine the rules after the test is run since, since depending on the metric, the testing platform, and your transactional data system, you might not have captured the KPI that you later determine would have been most effective to know.

-

“If you don’t know where you are going, any road will get you there.” – Lewis Carroll

-

In the end, the value of the research question is that it helps ensure all the effort and resources you invest in testing and optimization gets you to where you want to go. Or, as Phillip related, you might as well be testing through the looking glass…

Alice: Would you tell me, please, which way I ought to go from here?
The Cheshire Cat: That depends a good deal on where you want to get to
Alice: I don’t much care where.
The Cheshire Cat: Then it doesn’t much matter which way you go.
Alice: …so long as I get somewhere.
The Cheshire Cat: Oh, you’re sure to do that, if only you walk long enough.

-

Related Resources:

Research Update: The state of email marketing testing and optimization

Negative Lifts: Turning a 25% loss into a 141% increase in conversion – Thursday, December 1, 1 p.m. EST

MarketingSherpa 2012 Email Marketing Benchmark Report

Landing Page Optimization: How IBM applied homepage redesign learnings to landing page testing

What marketing errors are easiest to avoid? And how do we avoid making them?

My answer would be…those related to overconfidence. And, as to the second question, I’ll take the rest of this blog post to attempt to answer that.

-

Are you too confident?

In the business world, as in marketing, we usually look at confidence as a good thing. But the “overconfidence bias” can seriously harm your performance.

Here’s how Jonah Lehrer, an American journalist who writes on the topics of psychology and neuroscience, describes this overconfidence bias in The Science of Irrationality: A Nobelist explains our fondness for not thinking…

-

Consider the overconfidence bias, which drives many of our mistakes in decision-making. The best demonstration of the bias comes from the world of investing. Although many fund managers charge high fees to oversee stock portfolios, they routinely fail a basic test of skill: persistent achievement. As Mr. [Nobel Laureate, and professor of psychology at Princeton] Kahneman notes, the year-to-year correlation between the performance of the vast majority of funds is barely above zero, which suggests that most successful managers are banking on luck, not talent.

This shouldn’t be too surprising. The stock market is a case study in randomness, a system so complex that it’s impossible to predict. Nevertheless, professional investors routinely believe that they can see what others can’t. The end result is that they make far too many trades, with costly consequences.

-

Sound familiar? Is marketing a product any less complex than trading on the stock market?

-

How to bank on talent, not luck

I disagree with one aspect of Lehrer’s article. Not to put words into his mouth, but he seems to imply that there is no way to overcome the overconfidence bias. In marketing, I believe there is a way to do so (of course, perhaps that’s just me being …ahem … overconfident).

Let me explain what I mean, and let you be the judge… Read more…

In July, I wrote the blog post, Email Marketing Research: 7 steps for successful email marketing testing and optimization. In it, I discussed how continuous experimentation is the quickest path to peak performance. It enables marketers to go beyond best practices to learn what works for their organization and, more importantly, their customers.

I’m preaching to the choir, right? Well, I also encouraged readers to take the annual email benchmark survey conducted by MarketingExperiments’ sister company, MarketingSherpa.

Thankfully, this blog’s readers, along with more than 2,700 other email marketers, participated in the study. In appreciation, I would like to share with you the current state of email marketing testing practices.

Email testing on the rise

The number of marketers who routinely test email campaigns rose 3% from 2010 to 42%. This is good news as the industry inches closer to making it a prevailing practice.

Unfortunately, nearly six of 10 email marketing budgets do not have any money earmarked for testing and optimization. The majority (63%) of tests are conducted by employees for whom the practice is a part-time and secondary job responsibility, but still a formal part of their job description.

The minority includes the 23% of the email researchers who report the task as their primary focus and full-time duty and the 19% of marketers who perform experiments on the side without it being listed in their job description.

Testing practices most routinely performed

This information may help benchmark your programs and processes against the industry. But for this year’s survey, we wanted to delve deeper into which formal processes and guidelines organizations routinely use to test and optimize email campaigns. Here is a look into what we found.

Chart: More time needed for brainstorming and defining the testing objective

How routinely does your organization implement the following testing practices?

The above chart displays common testing practices in chronological order from top to bottom. We asked marketers to share with us which tasks their organizations routinely execute. The survey uncovered organizations are spending the most time segmenting their lists, understanding the impact of the test on the entire funnel, and documenting their findings. Read more…