|Wednesday, 21 September 2011|
Topic: Bad Data: The 3 validity threats that make your tests look conclusive (when they are deeply flawed)
If you’ve tuned into past MarketingExperiments Web clinics, you’ve seen some pretty impressive gains:
And we’ve shared transferable principles to help you achieve similar gains yourself.
But for this Web clinic, the MarketingExperiments team took a different approach and showed you not just what we did to get these types of results, but gave you an inside look at how we did it.
We didn’t get these results by only hiring marketing geniuses. We did it by developing a process focused on determining “what really works.”
Doing that takes much more than just plopping two landing pages into a platform with a splitter. You must know the results you’re seeing truly represent what is going on in the real world.
According to a survey in MarketingSherpa’s 2011 Landing Page Optimization Benchmark Report, at least 40% of marketers don’t even calculate statistical significance when they test.
This means at least 40% of marketers are probably making confident business and marketing decisions based on incomplete data that may be the result of random chance, not a true representation of buyer intent.
But beyond the math, there are three other validity threats that can make your tests look conclusive when they really aren’t.
So how can you be sure you’re making sound marketing decisions based on your tests?
In this Web clinic, Dr. Flint McGlaughlin explained the three validity threats that many marketers overlook when running tests, triggers to identify if they are threatening your tests, and how to mitigate the risks to the data you collect. Here are what some of the audience members had to say about the Web clinic: