Posts Tagged ‘metrics’

Website Optimization: Testing your navigation

September 11th, 2014 No comments

As we are testing our websites, we often focus on homepages, landing pages and funnels. These are the pages that “move the needle” and get results. However, there is one aspect of many sites that goes unnoticed by optimizers — the site navigation.

Site navigation is important because it gets your visitors where they need to be. Also, it’s usually one of the static elements of your site.

The navigation is visible on all of your pages and is often the one constant throughout the website.

It simply makes sense to focus your efforts on such a high visibility area that has such a great impact on your customers’ experience.

Now, you may be asking yourself, “What can I test in my navigation?”

To answer that question, I’ve constructed a short guide to help you start optimizing your navigation.

Potential navigation testing opportunities include:

  • Changing link names that may be confusing
  • Optimizing subcopy (if you give details in your navigation)
  • Changing hierarchies or organizations
  • Adding or deleting links
  • Optimizing visual features (icons)
  • Optimizing navigation indicators (hover and click functionality, lines, highlights, etc.)


Begin with goals and objectives 

It’s important to have clearly defined goals and objectives when testing your navigation.

While you want your site navigation to drive conversions, you should always remember that this is ultimately a tool for your site visitors.

It should guide them where they need to go in a clear, concise manner. So how do you measure your navigation’s success? What would be your primary KPI? In many tests, our KPIs are conversions or clickthroughs. However, much more thought must go into defining navigation KPIs.

Read more…

Marketing Analytics: 4 tips for productive conversations with your data analyst

September 19th, 2013 No comments

Every week, I encounter work orders from research analysts requesting data analysis on our Research Partners’ test results that are difficult to understand.

I think there’s a reason for this – poor communication.

I’ve noticed a lack of understanding of how data analysts think. Research analysts and many marketers do not define projects and goals the same way data analysts approach a data challenge.

Data analysis takes time and resources, so the less time spent interpreting desire over data will leave more room in the budget for necessary analysis. Better, clearer communication means everybody wins.

I wanted to share with you four tips to boost your team’s communication that will hopefully save you a little time and money in the process.


Tip #1. The more specific you are, the faster we can help you

If you’ve had experience working with data analysts, then you may know sometimes the conversation can be like asking someone for the time of day and they explain how their entire watch works, even if you just wanted to know the time.

But, who is really responsible for the failure to communicate here?

Is it the timekeeper for being specific, or was it the vagueness of the person who needed the time?

My point here is these kinds of communication mishaps often ring especially true in the analytics world. I can attest that an analyst with clear objectives and goals will be able to perform analysis at an accelerated rate with less revisions and meetings necessary to achieve results.

So, instead of asking for general analysis of a webpage, email campaign or other initiative, try asking for the specifics you want to know.

When an analyst hears general analysis, it’s like giving us a set of Legos and expecting us to instinctively know you wanted a plane constructed instead of the impressive 40-story futuristic building.


For example, let’s look at the following requests that highlight how just a few more details can make all the difference:

  • Request #1: “I need to know the clickthrough rate for new visitors compared to returning visitors.”

This question is going to get you what you need faster than asking for a general analysis of a webpage.

  • Request #2:I need to know the clickthrough rate for new visitors compared to returning visitors for the second to third step of the checkout funnel.”

The second request would likely deliver the rate from steps two and three for the different visitor types.

Now, if I only had the first request to work off of, I would deliver the clickthrough rate for every step of the funnel, which takes substantially longer and costs more.

This is also because data analysts have a tendency to flex their advanced analytics muscles when given the opportunity. We want to deliver quality work.

But, the time and effort to achieving those impressive results when something quick and easy would have been equally beneficial to your needs is a waste.


Tip #2. Knowing how you’re going to use the data helps

To better help you with a project, we need to know how you will use the data. 

So, when starting a new project, take some time beforehand to sit down with your analyst, it’s not as bad as you think, and discuss which specific topics or characteristics will help you gain the knowledge you need quickly.

If the data will be used for internal discovery, analysts will likely approach analysis, especially the final reporting, somewhat differently than for external reporting.


Tip #3. Creating fancy charts should be the exception, not the rule

Knowing how the data is going to be presented will help your analysts avoid wasting precious computational time making fancy charts and graphs if you only need the information for internal use.

Formatting of charts and graphs can end up taking way more time than one would imagine, so an analyst should worry about pretty charts only when needed.

Another reason it is important to discuss how the data will be used is because your analyst might use a more efficient reporting structure. They may use graph and chart types that you ask for when in fact, they could have used a more sophisticated technique if they knew what the end reporting needed to show an audience visually.

For instance, conversations that ask, “Do you need bar graphs for each individual variable?” should happen a lot more often than they do.  

This can become cumbersome and meticulous leading up to final presentations, but if the information is represented with clarity and efficiency using the right combination of charts, everyone wins.

Read more…

Marketing Metrics: Can you have one number to rule them all?

June 6th, 2013 3 comments

One of the common questions I receive from Research Partners focuses on what metric they should use to track and evaluate tests. The tendency is often to want a single metric that defines the measure of success.

While it is important to gather consensus on which key performance indicators, or KPIs, will be used to evaluate tests early, there should never be a reliance solely on a single metric as the gatekeeper of success given your secondary metrics can provide just as much – if not more – insight into your visitors behavior.


In the land of testing, the marketer with one metric is not king…

If you are only using one metric, you are not seeing a full picture. Each of your KPIs tells a part of the story of performance. Only relying on one alone can mislead marketers to make poorly informed decisions.

For example, let’s say you’re testing a PPC ad. As you know, the sole purpose of an ad is to get the click and let the landing page do the selling. For this reason, you determine  your KPI is clickthrough rate since that is what the ad directly affects.

Makes sense, right?

Now let’s say that your results come back and show that both ads receive the same number of clicks and that there is no statistically significant difference in clickthrough rate.

So what happens now?

Since clickthrough rate was the only metric measured, then you may draw the conclusion that both ads perform the same and that either could be used to achieve the same result and in some cases you may be right…

However, making this assumption is a big risk that flirts heavily with a similar risk of assumption derived from artificial optimization.

Read more…

Analytics & Testing: 3 statistical testing methods for building an advanced customer theory

May 30th, 2013 1 comment

When I was in college, I took a class on complex analysis and after all the lectures, studying and nerve-racking exams, I learned one important thing about customer behavior – some characteristics of a person will likely contribute to their future behavior.

In other words, my grandparents are not likely to start buying iPods, but at the same time my younger sister and her friends are not going to go out and start buying rotary telephones either.

Many times, variables such as gender, age, income, education and geographic location will likely play a role in why your customers say yes to your offers. This brings me to my point that selecting a test methodology robust enough to explore statistical relationships among variables is more important than ever to your marketing efforts.

In today’s MarketingExperiments blog post, we will simplify three basic testing models you can use to build an advanced customer theory.

Our goal is not to give you a Ph.D. in statistics, but rather, we want to provide you with a few test methods simplified and free of as much mathspeak as possible you can use to aid your team’s next discussion on test selection.


Test Method #1. ANOVA (Analysis of Variance)

Marketers can use the ANOVA testing method to understand if a statistical significance exists within or between groups. Landing page optimization is a good example of how ANOVA testing can used to analyze a customer’s response to different treatments based on variables of interest.

For example, suppose you’re testing landing pages and you want to determine if the income or education level of new and return customers has any statistical significance on the probability of conversion on the landing pages you’re testing, then ANOVA would be the optimal test method to consider using.


Test Method #2. Logistic regressions

Logistics is a testing method for prediction analysis. In other words, a logistic regression test can help you to discover the statistical likelihood of a conversion for customers in demographic A versus customers in demographic B.

With logistic regressions however there is only one catch …

Customers in both demographic groups A and B have to be known as significant contributors to the likelihood of conversions.


Test Method #3. Time series analysis

Time series analysis is a test method similar to logistical regressions in that’s it has a basis in predictive analysis, but time series analysis is focused on what you can learn from historical data trends.

Understanding the seasonality behavior of your website traffic is a perfect example of when you would use a time series analysis.

These are just a few of the testing methods available to help you learn more about your customers, but ultimately no marketer is an island. So, if you have a testing method that you use to build your customer theory, feel free to share it with in the comments below.


Related Resources:

Marketing Optimization: How to design split tests and multi-factorial tests

Marketing Metrics: Why all numbers aren’t created equal

How to Predict, with 90% Accuracy, Who Your Best Customers Will Be

Online Marketing Tests: A data analyst’s view of balancing risk and reward

Marketing Analytics: 6 simple steps for interpreting your data

November 7th, 2012 No comments

You’ve finally set up tracking on your site and have gathered weeks of information. You are now staring at your data saying, “Now what?”

Objectively interpreting your data can be extremely overwhelming and very difficult to do correctly … but it is essential.

The only thing worse than having no insights is having incorrect insights. The latter can be extremely costly to your business.

Use these six simple steps to help you effectively and correctly interpret your data.

Read more…

Validity Threats: How we could have missed a 31% increase in conversions

October 10th, 2012 No comments

Some marketers simply drop two landing pages into a split test tool, click a button, and then push live the page “winner” with the larger results number.

If you really want to benefit from split testing, you need to do a little more. You need a basic understanding of what you’re really doing when you’re testing. That includes validity, which is, at its most basic level, an assurance that the results from your tests actually reflect what is going on in the real world.

Let me show you an experiment with a MECLABS Research Partner in which an understanding of validity helped the team find a conversion lift that would have otherwise been missed.


Background: Consumer company that offers online brokerage services

Goal: To increase the volume of accounts created online

Primary research question: Which page design will generate the highest rate of conversion?

Test Design: A/B/C/D multi-factor split test



Click to enlarge

Read more…