It’s the same problem, every time…
Someone new to online testing, excited to get started, eager to begin crunching numbers and reporting amazing results:
- They decide on a few things that they instinctively think will raise conversions
- They make all the changes on one page, maybe two
- Setup all sorts of tracking that they’re not sure what to do with
- And push the start button thinking that their one page will see a major increase
Rarely does a test actually perform the way we expect it to, though. We believe that one set of pages will completely outperform another, when the reality is that some performance is all over the place.
When the test is complete, what will you report upstream? That the test was a wash? Or did you learn something that will bring you closer to the key insights that will have an effect on your bottom line?
Most beginners run into a test without being able to tell you what they are really testing.
They typically say, “I’m testing to see if this page increases conversions!” The problem with this statement is that it doesn’t do anything for you if the page, well, doesn’t.
In addition, if it does raise conversion, you don’t really have the slightest idea why. Instead, you get a false sense of confidence about what works on your site and what doesn’t. Then you get yourself into much deeper trouble, making assumptions for other parts of the site or customer segments that end up costing you revenue in the long term. Read more…