MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

During the Optimize your Email in Three Steps web clinic, I covered several measurement strategies to help you measure and prove the real value of your email campaigns. I was inundated with questions. Marketers are constantly in search of new “tricks” to find the perfect numbers that help them understand and tell the real story of their Internet marketing efforts.

While I was able to answer a few of these questions on Web Clinic Extra, I wanted to dive a little deeper today with some links and walkthroughs showing how to implement some of the metric items discussed. And please note, while these examples use Google Analytics, Omniture and many other companies have excellent tools with similar capabilities.

Tagging links within emails so you can measure email clicks within your Google Analytics

Requirements:

• Links tagged in email with Google Analytics tracking variables
• Destination Pages from the email with Google Analytics tracking code installed

Walkthrough:
First, with your emails, identify what links you want to track. For some people, just tracking CTA is enough, for others looking at additional navigational links (for example a supplied news article link or a support link) is also valuable data as well. Once you have compiled a list of links that you want to track, visit Google’s URL Builder Tool and start building your links. Please note that campaign source, medium, and name are minimum input requirements for this sort of tracking to work. You also have some remaining variables (name and content) you can use to insert segmentation data. In the example below, you will note that we inputted some demographic and business data:

Once you have built your links, insert them in the appropriate places in your email and hit the red button.

Please note, that using this tool is not necessary to build these links. Once you learn what variables are used, you can build a script that will automate this for you. You can then use internal databases of customer information to create dynamic and automated email tracking.

Also, once these emails go out, you can then create segments on these parameters and get targeted and segmented metrics for your email efforts:

As a final note, make sure you install Google Analytics on the page your audience will visit. This will be required to measure the clicks. Google Analytics tracking code is not required to be in the email, just the landing or website page they are landing on. The tracking script will read the URL variables that you put in your links in the email and recognize the data.

You can also apply these metrics to ecommerce and other reporting data within Google Analytics, giving you a further layer that attributes efforts to the bottom line.

How to incorporate form fields in goal reporting

Requirements:

• Adding the “onClick” markup JavaScript function in the form field you want to track
• Page must have Google Analytics tracking code installed

Walkthrough:
When I reviewed an example goal setup in the Optimize your Email in Three Steps web clinic, one of the steps I mentioned was a form field click as a goal step. In reviewing the clinic comments, I was stunned by the number of people that wanted to know how to do this and for me to explain further, so here we go.

First, as part of looking at email performance, many of us are sending users to pages that have form captures. For me, a great user experience or path to look at is users that click from the email, land on the target page, actually click into the form, and then submit/convert. So let’s look at a typical form code example, and how Google Analytics (GA) ties in:

Standard form input code example:

We can insert an onClick function to the form to capture when a user clicks into the field and complete the information. With this function we will be making a call to the GA tracking function: _trackPageview. What this function will do in our case is when a user clicks into the form field a page will be created in Google Analytics that we specify/create. For example purposes, with the page tracker function we will create the page /dec-email/form-field-email1.html.

After users have interacted with the form field, the /dec-email/form-field-email1.html will start to appear. Just to clarify, this page does not exist, but we have told GA to record clicks and interactions to the tagged form field to this mythological page we have made up. Also, if you are doing email testing, you could create a script that recognizes which email people are coming (e.g. URL variable) from and change this page dynamically as well. So instead of posting clicks to /dec-email/form-field-email1.html page, we use email2.html. Here is an example of Google Analytics markup on the form field:

Also, users tend to be quite “click happy” on forms, so make sure you look at unique visit data on these “pages,” rather than pageviews. Pageviews tend to be inflated because of this user behavior.

Lastly, once these “pages” are created in Google Analytics, you can insert them in goal funnels, just like other real pages. Your metrics will not skip a beat. Here is an example goal funnel that you could create in Google Analytics with the items we have covered:

**Make sure, if applicable, that you select the required step in your goal setup.

Leave a comment below and let me know the next measurement tricks you would like me to pull back the curtains on in future installments of the Magical Metrics Tour. Also, let me know if you find posts about custom or deeper metrics helpful.

For a more in-depth look at making email and social media deliver for your bottom line, check out Email Summit ’10 in Miami from January 20-22. PLUS, Dr. Flint McGlaughlin will teach a Pre-Summit Live Email Optimization Workshop to help you maximize your email capture rate and quality. Register by January 8 to receive an early bird discount of $200.

During our December 2 web clinic, Optimize your Email in Three Steps, Boris Grinkot, Heather Andruk, and Corey Trent answered questions from our audience about email relevance, frequency, and metrics.

We often don’t have time to answer all of our audience questions on the live web clinics. So we distilled all the questions into a few representative queries, and pulled Heath Andruk and Corey Trent in from the lab to share their insights on the latest edition of Web Clinic Extra:

Corey and Heather answered these questions:

Question 1 (1:05): In the frequency experiment shown on the clinic, was there variation in the types of emails (i.e. reminders, offers) in the low frequency email group (i.e. 1-4 per month) like there were in the higher frequency group?

Question 2 (1:45): What are some factors to determine good segments?

Question 3 (2:40): What do you do if you cannot segment?

Question 4 (5:00): Is there one age group more tolerable to frequent emails than others?

Question 5 (6:40): How do you figure out the best timing for emails?

Question 6 (9:55): Can you track goal pages that are outside of your domain with Google Analytics?

Question 7 (10:55): Can you track an email campaign in Google Analytics if you are sending emails with a 3rd party provider?

Come back to the blog on Friday for a technical addendum from Corey Trent. He has some specific tips to help you put his metrics wizardry into revenue-generating practice for your email campaigns.

The complete Flash version of the web clinic, along with a downloadable research brief (PDF), are now available on MarketingExperiments.com. If you have additional questions, use the comments section below or post them to our MarketingExperiments Optimization group.

During our November 11 web clinic, Surprising Wins from 2009: Using insights from an uncertain economy to drive 302% growth, Boris Grinkot, Adam Lapp, and Paul Clowe answered questions from our audience on a range of topics pertinent to our research from this year.

We often don’t have time to answer all the questions from our international audience on the live web clinics. So we’re launching a new feature on the blog – Web Clinic Extra. We distilled the best questions and posed them to Researchers Boris Grinkot and Adam Lapp for our first episode:

The complete Flash version of the web clinic, along with a downloadable research brief (PDF), are now available on MarketingExperiments.com. If you have additional questions, use the comments section below or post them to our MarketingExperiments Optimization group.

Senior Manager Boris Grinkot recently attended the Eloqua Experience 2009 Global User Conference in San Francisco. He sat down to answer a few questions about his key takeaways for online marketers…

Q: You were the only non-Eloqua employee to have a booth in the Marketing Effectiveness Zone. Users from around the world sought your advice about conversion rate optimization. Was there any pattern to the challenges marketers brought you?

Most websites are not really having a conversation with the customer. They are not guiding the customer through the page. And, most importantly, they are not testing their pages. They may test an email message or a form with Eloqua, but they don’t test the whole page.

There is massive room for improvement to optimize landing pages to ensure they are customer-focused, and then continually improve the results generated by those pages through testing.

Q: Measuring those results was also a major topic of conversation at Eloqua Experience?

Yes. Eloqua is seeking to standardize key performance indicators, or KPIs, that CxOs use to measure results in much the same way financial accounting uses key ratios and cash flow statements, income statements, and balance sheets. Today, most measurements in web analytics – like visitors per month, for example – do not directly translate to a standard accounting metric, such as net revenue.

Q: Based on past statements we’ve received from web clinics and here on the blog, KPIs are a little misunderstood by some marketers. How can readers choose and use KPIs effectively?

In our research, we find marketers most frequently tend to look at numbers like conversion rate, bounce rate or number of visitors. While these numbers can be meaningful within the context of page or process functional performance, they don’t necessarily do a good job of measuring the financial performance.

Revenue per visitor is usually an essential KPI that connects online customer behavior to a financial outcome. While this is a more straightforward KPI to calculate for ecommerce sites, even if you have a lead-generation site, you should understand the value of each lead to determine your revenue per visitor.

Average order size can also be a meaningful KPI that helps you distinguish customer segments (e.g., collectors vs. gift shoppers) or test functional changes like in-cart upsells. Choosing the right KPIs is a big topic, but the short of it is that you need to distinguish between behavioral, demographic, and financial metrics and use them appropriately.

Q: So the important thing to remember is, that while some metrics might be useful in an intermediate step, the overall goal should be a revenue-based number?

Absolutely. For testing and optimization, you need those intermediate numbers.

For example, if you’re optimizing just one step of a shopping cart, conversion rate or clickthrough are important testing metrics. But you don’t want to lose sight of the big picture – which would be overall revenue or revenue per visitor. Let’s say you’ve optimized a step and more visitors are clicking through, but they are less motivated and in the end are buying less. If you don’t have that overall revenue KPI, and you just looked at conversion rate, you would erroneously assume that you have definitively improved your shopping cart.

Q: What are some other caveats when choosing a KPI?

Even if you choose the right KPI, you can still get bad information by not looking at individual channels. You may have good revenue per visitor, but that number is just an average. What if one channel is delivering ten times the revenue per visitor of another channel? That is important information to have, especially if you’re paying for traffic, because you need to understand how that spend converts to revenue.

Q: What is the overall benefit of choosing the right KPI for C-level reporting?

By focusing on dollars instead of traffic, business leaders gain a deep understanding about how the investments they make impact revenue generated through the website. If you invest more in a site, what ROI are you getting?

Clickthroughs and conversion rates only muddy the answer to this question. You need numbers directly related to the spend of each campaign. And not in the aggregate, but specific to a channel – so you know that investing X in PPC, email, TV, or even print will lead to a return of Y.

The other upside, which I’m glad Eloqua is pushing, is that numbers like overall revenue, revenue per visitor, or cost per acquisition bring web metrics much closer to standard accounting. These numbers give CxOs the best indicators of their site’s performance in a format they are used to seeing.

Use the comments section below or post your questions to our MarketingExperiments Optimization group.

Editor’s note: During our October 28 web clinic about properly guiding visitors to your conversion objectives, researchers Boris Grinkot, Corey Trent, and Heather Andruk fielded several audience questions.

Q: Does left navigation diminish conversion rate?

It all depends on the purpose of the page. For a simple landing page to which you drive prospects from PPC ads, affiliates and banner ads, this additional navigation will likely distract visitors and drive down conversion rates. It ends up being just one more competing objective. The answer to the “What can I do?” question now has multiple answers.

Of course, there are degrees of distraction. Deemphasized navigation with supportive elements such as FAQ is less distracting than very prominent navigation with several drop-downs and hover-over pop-ups.

For a page that is an integral part of your site, navigation is often essential to help guide your users through the overall site. If there is navigation on every other page, having one or two pages without navigation will cause a disconnect and you will likely lose the visitor.

Overall, you must remember that these are just guiding principles developed from our years of research. In the end, the best answer is to test these elements to see how they affect conversion on your specific pages.

Q: How do you know when to offer competing products?

There is no single answer, but we can give you a few good test ideas. If you have secondary products that do not compete with your primary offer, you may try to offer them before a purchase is made. If you have auxiliary offers that may distract from the primary offer, wait until the purchase is complete to offer the upsell. An excellent example is Amazon.com, which is a master at upselling additional products both in the cart and after purchase.

To decide what to test, look at your metrics to see where people enter and exit your site. For example, do they hunt for different features or go straight to purchasing a product? From these metrics, try to decipher what products users may be looking for in addition to what you are currently offering. The metrics can provide actual data to back up your decision about which competing products you want to test, and where you want to offer them.

Q: Do the five elements of directing visitor eyepath apply to B2B sites with long sales cycles?

These are tactical recommendations meant to help guide your visitors’ thought process through a page. The specific product or offer does not matter, since we are not trying to optimize a page; rather optimize the thought process of the visitors to a page. So using shape and color to emphasize your key points, and size to draw attention to your headline only helps more effectively express your value proposition in the conversation you are having with your customer.

Therefore, these elements apply to any page and it is worth noting that they may be especially important for B2B sites. This audience tends to scan pages more frequently than consumer audiences, so using color and shape to help emphasize key points in a way that is easier for your audience to digest is especially relevant.

Q: Are there any tools that can help me select colors?

Yes. Kuler.adobe.com and colorschemedesigner.com can help you choose color palettes.

Q: Should images have humans in them?

It depends on what you’re trying to accomplish with the image. For some products, it makes more sense to show a point-of-view orientation to give visitors the sensory feel of what it is like to use the product, such as showing the driver’s view from a car or the view from a hotel. For others, you need people in the picture to show the product being used.

When you do use people, having a good understanding of your demographic is extremely useful. Use images that your visitors can relate to (e.g., young, middle aged, seniors).

Images of people will also help guide visitor eyepath. The image should be “looking” at where you want the user to go. So for example, if you have an image on the left and copy on the right, be sure the person in the image is looking to the right to direct the user’s eyepath towards the copy.

As you test your pages, keep in mind how powerful images are. In the initial one or two seconds many visitors take to judge your page, your images can connect with them and draw them in or repel and cause them to bounce. Be careful in using stock photography, as consumers are becoming increasingly savvy to its use and may consider it to be misleading – especially if used with believability elements such as a testimonial.

Have additional questions? Other things you’d like to Ask an Optimizer? Use the comments section below or post your questions to our MarketingExperiments Optimization group.

Editor’s note: During our October 14 web clinic on overcoming testing obstacles, senior researchers Boris Grinkot and Gaby Diaz fielded several audience questions on optimization testing. We’ve distilled the best questions and answers for the latest edition of our Ask an Optimizer column.

Q: How long should we split test a variable or collect data points in order to make a valid decision?

You need to use a statistical formula called the chi-square test that takes into account the number of samples you are going to collect, the amount of variance (between the control and the best performing treatment), and the statistical confidence level you desire before making a business decision (typically 95%).

Since you won’t know the variance before you start the test, you must project it to begin with and then plug the actual numbers into the formula as you conduct the test. The number of samples will be affected by the variance. The larger the variance, the fewer samples you need to collect to reach a statistically significant conclusion.

To learn more about testing, consider taking our The Fundamentals of Online Testing course, which includes our proprietary test protocol.

Q: How many tests can you have going on at the same time without skewing your results?

You can run as many multiple tests at the same time as you want, as long as they are independent and do not impact each other.

Q: What are the benefits of multivariate testing vs. A/B testing in marketing?

Multivariate testing has become quite a buzzword, so instead of focusing on the benefits, let’s discuss some things you have to take into account for a successful test. A multivariate test will be heavily impacted by some obstacles we mentioned before – time, IT resources, availability of analytics, etc.

Mainly, you need a substantial amount of testing to reach statistically significant results. On the testing web clinic, we talked about radical redesigns as a solution to a time crunch. But if you have the time, you can run a multivariate test to reach intermediate conclusions before drilling deeper with A/B split tests.

How much time do you need? As in question one, duration is affected by how many samples you have and the number of treatments. The more treatments, the more samples you have to get and the longer the test needs to run.

For example, in a single factorial A/B split test, your traffic is split just in two ways. So if you have 100,000 visitors, 50,000 see the control and 50,000 see the treatment. But if you have three headlines, three versions of the body copy, and three versions of the call to action, that becomes a fairly large number of combinations and will split your test up many more ways so it will take that much more time to get a statistically valid result.

Once a multivariate test helps you determine which elements had the biggest influence, A/B split tests will help you optimize further. For example, if you see the headline had a big influence in a multivariate test, you could perform single factorial tests to see which headline works best.

But keep in mind, on its own a multivariate test can be fairly hard to interpret. While you may find a certain combination performs well, it can be difficult to determine which of multiple variables to attribute those gains to. It is best to use multivariate testing first, then take your best guess about which elements to drill deeper into and test further.

You’ll also want to review three past research briefs that cover these areas in more depth: Multivariable Testing, A/B Split Testing, Conversion Rate Optimization Tested.

Q: How do I test two different pages with two different channels? For example, two pages with my new customers and two pages with my returning customers?

By using cookies, you can identify which customers are new and which are returning. Since returning visitors are already highly motivated and can overcome high levels of friction, you might not even want to show them a homepage at all. Perhaps show them their latest purchases or a recommendation page suggesting related items based on what they’ve purchased in the past.

Keep in mind, as we saw with the web clinic‘s second case study, it was much harder to achieve gains from the shopping cart traffic since it was already converting so high. You likely have much more room to gain from your new visitors. On the other hand, even small gains in your highest converting channels could provide a major boost to the bottom line. Testing different channels will help you understand what works best for each of them.

Q: I’ve been running a test for over a month and haven’t seen a statistical significance. Should I keep running the test or stop?

You should stop. Month-to-month variations in your traffic, due to seasonality for example, will start to skew the results. Reassess the page and launch a new test.

Q: How do you test for customer loyalty to a website?

You have to get really creative with your data analytics. One of the obstacles is prioritization. First, identify your objective. That may seem obvious ­– “my priority is to increase customer loyalty.”

But you have to drill deeper to understand how to measure customer loyalty to determine your key performance indicator (KPI). For example, one measurement might be return visits to your site. Working with a research partner recently, this was an important performance indicator. So KPI would be the number of visits per month from the same visitors (unique visits divided by total visits per month).

For this example KPI, you can run a test to determine which treatment is going to get people coming back to your site. You could test by sending target emails that remind them of reasons to return. If you have a retail site, perhaps the emails highlight a sales promotion. If it’s a service site, highlight an action they need to take on the site ­– like completing a form.

We teach a whole separate course on email optimization. But at a high level, the goal of an email is to get them to click back to your website. You don’t need all the information in the email itself. Get them to come back. And then you can test and measure the loyalty in terms of return visits.

Q: Can you test different channels with Google Analytics?

Yes. Combined with Google Website Optimizer, you can test different channels. Google Website Optimizer will track the conversion rate and statistical significance. But if you have unique URLs and mark your page with Google Analytics, you can see how they’re working in combination with your website and see the type of visitors you’re getting.

You can also see success metrics such as the conversion rate and where people navigate after that page. You can set up tests for channels such as PPC, banner advertising, or any other channel. Also, see our recent post on how to use the new features in Google Analytics.

Have additional questions? Other things you’d like to Ask an Optimizer? Use the comments section below or post your questions to our MarketingExperiments Optimization group.