Erin Fagin

Marketing Metrics: Can you have one number to rule them all?

June 6th, 2013

One of the common questions I receive from Research Partners focuses on what metric they should use to track and evaluate tests. The tendency is often to want a single metric that defines the measure of success.

While it is important to gather consensus on which key performance indicators, or KPIs, will be used to evaluate tests early, there should never be a reliance solely on a single metric as the gatekeeper of success given your secondary metrics can provide just as much – if not more – insight into your visitors behavior.


In the land of testing, the marketer with one metric is not king…

If you are only using one metric, you are not seeing a full picture. Each of your KPIs tells a part of the story of performance. Only relying on one alone can mislead marketers to make poorly informed decisions.

For example, let’s say you’re testing a PPC ad. As you know, the sole purpose of an ad is to get the click and let the landing page do the selling. For this reason, you determine  your KPI is clickthrough rate since that is what the ad directly affects.

Makes sense, right?

Now let’s say that your results come back and show that both ads receive the same number of clicks and that there is no statistically significant difference in clickthrough rate.

So what happens now?

Since clickthrough rate was the only metric measured, then you may draw the conclusion that both ads perform the same and that either could be used to achieve the same result and in some cases you may be right…

However, making this assumption is a big risk that flirts heavily with a similar risk of assumption derived from artificial optimization.

What metrics and methodology can tell you about your customers

At MarketingExperiments, the element of motivation is given the most weight in our methodology for a reason – it can have a dramatic impact on performance that is not always apparent at first.

So while both ads had the same clickthrough, one may have attracted a group that was more qualified or motivated to move through the remainder of your sales funnel. Had only the clickthrough rate been used to analyze performance, it’s likely that valuable customer insights would have been missed and ROI would have been left on the table.

The big caveat here – don’t fall into the trap of assuming variables in your sales funnel that remain constant will continue to perform at a constant.


Related Resources:

Analytics & Testing: 3 statistical testing methods for building an advanced customer theory

Analytics and Testing: 3 tips to optimize your testing efforts

Marketing Optimization: The 3 phases of evidence-based marketing

Erin Fagin

About Erin Fagin

Erin Fagin, Senior Research Manager, MECLABS Institute Erin works daily to help Research Partners with testing and optimization strategies. Prior to joining MECLABS, Erin worked in a research internship in Florida’s tourism industry. Erin holds her B.S. in advertising and management from Florida State University. When Erin’s not working, she loves to read, travel and play sports.

Categories: Analytics & Testing Tags: , , , , ,

  1. June 6th, 2013 at 19:33 | #1

    Makes a lot of sense, Erin. I’ve recently seen how this applies to keyword research as well. Checking several different tools gives a far more accurate picture of the situation. And when you’re able to improve the quality of data you’re basing your optimization on, it’s a no-brainer. Makes everything so much more effective and efficient. Thanks for the article! (Oh, and was the title a subtle LOTR reference? 🙂

  2. June 9th, 2013 at 15:43 | #2

    Marketing is incomplete without new strategies and techniques which are being used these days. Using keywords and creating quality content is very important when it comes to making your online presence.

  3. June 10th, 2013 at 11:33 | #3

    I’ve always believed, “The Devil’s in the details”.

We no longer accept comments on the MarketingExperiments blog, but we'd love to hear what you've learned about customer-first marketing. Send us a Letter to the Editor to share your story.