Do a quick Google search on “things to a/b test on a website,” scan the results for a moment, then come back and read the rest of this article.
Most of you reading this are marketers, so you know I’m taking a big risk by telling you to go do something else before you read my article.
In fact if you’re reading this now, you’re probably one of the very few who made it back from that incredibly distracting activity I had you do. Thank you. You are exactly the person I want to be reading this. The others can go on their merry way. They are not the ones who need to hear this.
I had you do that search because the Internet is full of people telling you to test things on your website such as color, button size, layouts, forms, etc. I wanted you to get an idea for what’s out there.
Now, I want you to understand why almost everyone writing those articles is wrong …
… or at the very least, missing the point.
Please don’t view this as me putting down the people who wrote those articles. I know a few of them personally, and I highly respect the work they are doing. This is not about whether their work is good or bad.
I’ve personally written many articles exactly like the ones they’re writing. In fact, they have one up on me because at least their articles are ranking in Google for popular search terms.
The reason they are missing the point is that most of those articles are focused on the elements of a page rather than the serving of a customer.
I get why they do it.
Webpages are far easier to understand than people. Webpages are a collection of 0s and 1s. People are a collection of who knows what.
And most of you, readers, are looking for webpage fixes — not a deeper, fuller way to serve your customer.
There is nothing necessarily wrong with you, but it’s just that we naturally focus on our own self-interest. It isn’t wrong in itself.
What is wrong is the methods we use to achieve our own goals. I don’t mean morally wrong. I mean practically wrong.
Our objective should always be: Make as much money possible.
MECLABS Institute has found after more than 15 years of research that the best method for achieving this objective is to spend as much money possible on serving your customer.
Until we can view every A/B test we run as an opportunity to better serve our customers, we will just be running (ultimately) ineffective tests on page elements.
It doesn’t really matter in the long run which color, layout or page element is going to perform well.
The Internet is constantly changing. Design trends are always going to influence how we look at webpages and their elements. What matters for marketers in the long run is how well we understand and, consequently, how well we can serve our customers.
This is also why he often says, “The goal of a test is not to get a lift, but rather to get a learning.”
However, it’s one thing to hear this, another to really understand what it means.
It really means we want to conduct research, not run a test.
We want to learn a tangible lesson about our customer so that we can apply it to other areas of our marketing and achieve a maximum return on the amount of time and energy we spend on testing.
Let me show you what I mean with a real-world example. Here’s what happens when you just run an A/B test that is focused on a page element. Let’s take color for instance.
You have two treatments. The only thing changed is the background color.
You also have a result. In this case, the result was a 19.5% increase in clickthrough at a 92% level of confidence. But here’s where things get tricky.