Twenty four conversations, countless emails and three-and-a-half Red Bulls later, I walked away from my last coaching clinic session at MarketingSherpa Email Summit 2013.
I realized a lot of the questions I was asked were deeper than:
Instead, at the root of many questions was a thirst for advice on strategic thinking and test planning.
A common problem I see among marketers is a lack of sound strategy behind testing. We are asking the wrong questions. We’re asking, “What headline should we test?” when we need to begin by asking, “Why is the current headline underperforming?” We need to ask “Why?” before “What?”
As an optimization manager, one of my responsibilities is guiding my coworkers through planning an effective test series for our Research Partners. Each time I go through this exercise, there is a thought process I follow, which is what today’s MarketingExperiments blog post will teach.
But before we begin, I want to emphasize the most important point of this post – the reason I follow this process. If you leave this page with nothing else, please remember:
We test solutions to problems, not ideas.
Okay, now we can get down to the nitty-gritty. Let’s talk about the thought process you should follow to craft a strategy-centered test.
STEP #1: STATE YOUR GOAL
This will lay the foundation for your test, and you will need to continuously refer to your goal. It helps me to write the goal on the whiteboard to remind everyone what we are working towards. This way, we all stay focused.
An example of your goal could be to:
Increase the average order value of customer transactions.
STEP #2: IDENTIFY THE PROBLEM – The “Why”
Now that you have identified your goal, it’s time to start thinking about how to achieve that goal. But before you can craft a successful solution, you need to understand the problem. Ask yourself, “Why?”
If your goal is to increase average order value, you may begin by asking:
Why are users only spending an average of $50 on my site?
Once you have asked “Why?” you should then form your hypothesis by formulating possible answers to your question – the “Because.”
Users are only spending an average of $50 because:
A. We place the most emphasis on the product that costs $50.
B. The only product that is relevant to our customers is the one that costs $50.
C. They are unaware of the extra benefits of our more expensive products.
As you can see, you can quickly identify multiple possible problems. A trap many people fall into at this point is trying to solve all of these problems at once. This will put you on the fast track to failure with no tangible learnings from your test. Choose one problem to solve at a time.
Sometimes, the biggest problem may not be feasibly solved within your project scope (possibly Problem B). You must choose the problem to solve by weighing the benefits of solving the problem with the costs of testing and implementing the solution.
Let’s move forward with Problem A – We place most emphasis on the $50 product.