Archive for December, 2009

Creating a Culture of Testing: How to defeat the tyranny of best practices

December 14th, 2009

You can hear Senior Manager of Research Partnerships Andy Mott answer the question How Can You Make Your Web Site Smarter? on the replay of Omniture’s latest webinar. But in my experience with these events, there is usually an interesting back story. So I cornered Andy in his office at a vulnerable time (his beloved Gators had recently lost the SEC Championship game) and found out what he really wanted to say…


Q: You discussed the 2009 Omniture Online Conversion Survey on a recent Omniture webinar. What surprised you the most?

Well I won’t say this surprised me. Maybe saddened is a better word. The survey asked “How frequently is online marketing testing employed in your company?” About half of the respondents said infrequently or never.


Q: Wow! That is pretty shocking, especially considering that these people are already familiar with testing through Omniture or MarketingExperiments. Maybe I could understand if this were the general population of marketers. But why have testing tools in place and not test? Why do you think half of them are flying blind?

Well people know they need to test. They probably know their competitors are testing and getting results. But the idea of executing a test is such a paradigm shift in the way that they’ve always done things.

Those that are higher in an organization tend to be more experienced. And if they are more experienced, they may be locked into the advertising agency way of doing things from 30 years ago, just like the doctor who overlooks recent findings and does what worked best for him when he went to medical school.


Q: Change is difficult. But still, thirty years ago these same people were also wearing polyester and doing the hustle. I’m a little skeptical that they would still try to shoehorn old media principles into new media.

It’s not intentional. If something has always worked for you, why change?

But what we really have is the tyranny of best practices. I’ll give you a great example. Many marketers still believe that they must have the call to action “above the fold” on a web page. Yet testing has shown this to be an utter myth.


Q: And nothing disproves a best practice better than a test that shows what actually works for their specific situation.

That’s the thing. Once companies start testing and see the ROI they are absolutely hooked.



Q: How do you take that first step? For, say, an email marketing manager reading this, how do you create a culture of testing in an organization?

Business-level executives don’t care about optimization or testing or even online marketing really. What they care about is results. So you need to talk to them in their language.

At MarketingExperiments, we publish all of our research and it is available for free. So go to the research archive and pull some experiments so you can show example results and make the business case for testing. At this point, all you are looking for is a small budget to begin testing.

Those first tests will help you establish a beachhead that you can use to further penetrate the organization. Because once businesses see the results they can gain from testing, it can get addictive. It’s like eating chips while watching a football game, you just can’t stop.


Q: The challenge is to just get the ball rolling. This sounds great in theory. Do you have any real-world examples?

I have countless examples. Since we started this conversion by talking about my recent webinar with Omniture, let me tell you about a Research Partner that first got interested in testing by attending an Omniture webinar that featured Dr. Flint McGlaughlin.

Companies that test usually like to stay anonymous because they view this process as such a competitive advantage. So I’ll just say they are a very large financial institution.

So this marketing manager attended Flint’s webinar and was totally sold. He was convinced that they should begin testing. But he’s only responsible for a very small patch in this giant company. It took him six months to get the approval to begin testing, doing the things I previously mentioned.


Q: Six months? It’s easy to get discouraged in that time. I’m not sure how many people would see it through.

But here’s the kicker. That marketing manager and his boss are now charged with trumpeting this win across the entire organization. He is now in front of his boss’s boss’s boss presenting his test results. In fact, in a few days he will be presenting in front of the SVP committee that advises the CEO.


Q: Well then he must have achieved some really out-of-this-world test results. What did he get…three digits…four digits? I mean, how common is that?

They got a 38% revenue boost over what the agency was doing.


Q: Well, that sounds decent, but a committee of SVPs really cares about 38% in one test?

You say that because you are so used to the power of testing, so you just want to see huge numbers. Let me put this another way – by not testing they would have been leaving 38% more money on the table since the cost of testing was infinitesimal compared to their massive marketing budget.

And that’s the thing. This company has a huge marketing budget. They sponsor the Olympics. They name stadiums. They purchase a ton of media. And since they don’t have space to sell in most of these executions, they’re driving everyone to the website. So if they find they could make more 38% more money without having to increase any of these huge marketing spends, the increase in ROI is humongous. Even a one or two percent increase could make or break a quarter.


Q: I see. I didn’t make the connection to that old media marketing spend. But I would think it goes beyond just old media driving people to a website. Online marketing is growing by leaps and bounds. I would think companies want to make sure they are getting a return on that investment as well. According to Forrester Research, digital spending will nearly double over the next five years at the expense of traditional marketing.

Forget five years from now, even today companies spend more than $25 billion on interactive marketing – things like mobile marketing, social media, email marketing, display advertising, and search marketing. That is 12% of all advertising spending. So when enterprises, like that financial institution I discussed, learn that they can take just a tiny fraction of the spend on this growing segment and invest it in a way that ensures the effectiveness of everything else they do – with real-world, statistically valid data – they get very excited.


Q: And I would think, for the employees that can tell management “I know how to get the best ROI from this” – not think or have an opinion, but know with real numbers – that’s quite a smart career move.

If other people are discussing so-called “best practices” and you’re showing real results, then you become the go-to person. The one who knows how this stuff really works. Because nothing defeats the tyranny of best practices as well as the audacity of testing.

And if you’re the guy that knows the right things to do in an explosively growing field like Internet marketing, while marketing budgets on everything else are falling, you’re in a good place no matter what the economy is doing.

How did you get the ball rolling on testing in your organization? What are your biggest challenges to create a culture of testing in your organization? Share your triumphs and challenges in the comments section below or post them to our MarketingExperiments Optimization group.

The Magical Metrics Tour: Demystifying the secrets behind analytical “tricks” to help you drive ROI

December 11th, 2009

During the Optimize your Email in Three Steps web clinic, I covered several measurement strategies to help you measure and prove the real value of your email campaigns. I was inundated with questions. Marketers are constantly in search of new “tricks” to find the perfect numbers that help them understand and tell the real story of their Internet marketing efforts.

While I was able to answer a few of these questions on Web Clinic Extra, I wanted to dive a little deeper today with some links and walkthroughs showing how to implement some of the metric items discussed. And please note, while these examples use Google Analytics, Omniture and many other companies have excellent tools with similar capabilities.

Tagging links within emails so you can measure email clicks within your Google Analytics


  • Links tagged in email with Google Analytics tracking variables
  • Destination Pages from the email with Google Analytics tracking code installed

First, with your emails, identify what links you want to track. For some people, just tracking CTA is enough, for others looking at additional navigational links (for example a supplied news article link or a support link) is also valuable data as well. Once you have compiled a list of links that you want to track, visit Google’s URL Builder Tool and start building your links. Please note that campaign source, medium, and name are minimum input requirements for this sort of tracking to work. You also have some remaining variables (name and content) you can use to insert segmentation data. In the example below, you will note that we inputted some demographic and business data:

Tool: URL Builder

Once you have built your links, insert them in the appropriate places in your email and hit the red button.

Please note, that using this tool is not necessary to build these links. Once you learn what variables are used, you can build a script that will automate this for you. You can then use internal databases of customer information to create dynamic and automated email tracking.

Also, once these emails go out, you can then create segments on these parameters and get targeted and segmented metrics for your email efforts:

Google Analytics ROI Revolution

As a final note, make sure you install Google Analytics on the page your audience will visit. This will be required to measure the clicks. Google Analytics tracking code is not required to be in the email, just the landing or website page they are landing on. The tracking script will read the URL variables that you put in your links in the email and recognize the data.

You can also apply these metrics to ecommerce and other reporting data within Google Analytics, giving you a further layer that attributes efforts to the bottom line.

How to incorporate form fields in goal reporting


  • Adding the “onClick” markup JavaScript function in the form field you want to track
  • Page must have Google Analytics tracking code installed

When I reviewed an example goal setup in the Optimize your Email in Three Steps web clinic, one of the steps I mentioned was a form field click as a goal step. In reviewing the clinic comments, I was stunned by the number of people that wanted to know how to do this and for me to explain further, so here we go.

First, as part of looking at email performance, many of us are sending users to pages that have form captures. For me, a great user experience or path to look at is users that click from the email, land on the target page, actually click into the form, and then submit/convert. So let’s look at a typical form code example, and how Google Analytics (GA) ties in:

Standard form input code example:

<input type="text" name="emailaddress" size="16" /><br />

We can insert an onClick function to the form to capture when a user clicks into the field and complete the information. With this function we will be making a call to the GA tracking function: _trackPageview. What this function will do in our case is when a user clicks into the form field a page will be created in Google Analytics that we specify/create. For example purposes, with the page tracker function we will create the page /dec-email/form-field-email1.html.

After users have interacted with the form field, the /dec-email/form-field-email1.html will start to appear. Just to clarify, this page does not exist, but we have told GA to record clicks and interactions to the tagged form field to this mythological page we have made up. Also, if you are doing email testing, you could create a script that recognizes which email people are coming (e.g. URL variable) from and change this page dynamically as well. So instead of posting clicks to /dec-email/form-field-email1.html page, we use email2.html. Here is an example of Google Analytics markup on the form field:

<input type="text" name="emailaddress" size="16" onClick="javascript:pageTracker._trackPageview('/dec-email/form-field-email1.html');" /><br />

Also, users tend to be quite “click happy” on forms, so make sure you look at unique visit data on these “pages,” rather than pageviews. Pageviews tend to be inflated because of this user behavior.

Lastly, once these “pages” are created in Google Analytics, you can insert them in goal funnels, just like other real pages. Your metrics will not skip a beat. Here is an example goal funnel that you could create in Google Analytics with the items we have covered:

Step One: /dec-email/index.html?id=email1
Step Two:
(_trackPageview created page)
Goal URL: /dec-email/thank-you.html

**Make sure, if applicable, that you select the required step in your goal setup.

Leave a comment below and let me know the next measurement tricks you would like me to pull back the curtains on in future installments of the Magical Metrics Tour. Also, let me know if you find posts about custom or deeper metrics helpful.

For a more in-depth look at making email and social media deliver for your bottom line, check out Email Summit ’10 in Miami from January 20-22. PLUS, Dr. Flint McGlaughlin will teach a Pre-Summit Live Email Optimization Workshop to help you maximize your email capture rate and quality. Register by January 8 to receive an early bird discount of $200.

Web Clinic Extra: Optimize your Email in Three Steps

December 9th, 2009

During our December 2 web clinic, Optimize your Email in Three Steps, Boris Grinkot, Heather Andruk, and Corey Trent answered questions from our audience about email relevance, frequency, and metrics.

We often don’t have time to answer all of our audience questions on the live web clinics. So we distilled all the questions into a few representative queries, and pulled Heath Andruk and Corey Trent in from the lab to share their insights on the latest edition of Web Clinic Extra:

Corey and Heather answered these questions:

Question 1 (1:05): In the frequency experiment shown on the clinic, was there variation in the types of emails (i.e. reminders, offers) in the low frequency email group (i.e. 1-4 per month) like there were in the higher frequency group?

Question 2 (1:45): What are some factors to determine good segments?

Question 3 (2:40): What do you do if you cannot segment?

Question 4 (5:00): Is there one age group more tolerable to frequent emails than others?

Question 5 (6:40): How do you figure out the best timing for emails?

Question 6 (9:55): Can you track goal pages that are outside of your domain with Google Analytics?

Question 7 (10:55): Can you track an email campaign in Google Analytics if you are sending emails with a 3rd party provider?

Come back to the blog on Friday for a technical addendum from Corey Trent. He has some specific tips to help you put his metrics wizardry into revenue-generating practice for your email campaigns.

The complete Flash version of the web clinic, along with a downloadable research brief (PDF), are now available on If you have additional questions, use the comments section below or post them to our MarketingExperiments Optimization group.

Conversion Diagnosis: Toyota Material Handling Nederland

December 7th, 2009

On December 3rd in Haarlem, The Netherlands, Dr. Flint McGlaughlin spoke at the Dutch Email Marketing Association Summit, also known as the “Sexy” Email Event. The Director of MECLABS (the parent company of MarketingExperiments) discussed how to improve email and related landing page conversions and conducted live optimization of audience submissions. Below is one of those submissions, along with a conversion diagnosis that will hopefully give you some ideas to improve the performance of your own marketing efforts. Please note, it has been translated from its original language of Dutch.

This submission is a B2B website seeking to drive downloads of a whitepaper.

Toyota[click to enlarge image]

For this particular page we have to assume that visitors are well qualified. They either have searched for “electric pallet trucks” or have navigated through the site to arrive here. Knowing this, the headline is pretty standard. It’s effective in several ways:
• Continuity between steps
• Communicates “where” visitors are

However, it does not answer two key questions:
• What can I do here? I don’t know if I can order, request more info, get a quote, or just view photos.
• Why should I order a pallet truck from Toyota instead of another competitor?

Several variations of the headline that add value and provide the visitor with guidance should be tested.

Value: You must add appeal, exclusivity, and credibility to give force to the value proposition of this product. Consider testing a quantitative variation such as: “Electric Pallet Trucks: 95% Customer Approval Rating” or “Electric Pallet Trucks: Crafted by Toyota for More Than 50 Years.”

Guidance: You must greet the visitor and “hold their hand” as they experience the page. In the primary headline, communicate value. But then in the sub-headline, you want to make it what exactly they can do on this page clear. For instance, “Download Product Details and Get Price Quotes.”

Once a visitor reads the headline, they are then forced to digest a bulky paragraph of six lines…a hard swallow. Most likely, your typical visitor may read the first or second line then have their eye-path drawn away from the paragraph by the large images. If you have important information in the last few lines, it will be missed. We recommend using a maximum of two-to-three lines of copy so that it’s easy to get to the point and move on to the next paragraph.

Also, there are no bolded words in the copy. This creates a disruption on the page that halts the eye-path and visitors just see one large chunk. Instead of moving seamlessly down the page, visitors may get lost in the copy. Important words such as “ergonomic” don’t stand out from trivial words such as “things.” It all just runs together.

You should also bold keywords so that your page adapts to different visitor segments. People who don’t like to read and just want to get to the point can just scan four words and move on. While people who need every single detail can still take their time reading the copy.

Where the heck do I click? Okay, so it looks like clicking on the triangles results in a whitepaper download. But they don’t appear clickable and they blend in with the images. This page should definitely make links that are directly below the images into buttons and ensure they have properties that make them appear clickable – such as bevel and drop shadow. Also, test button copy that is clear and provides a tangible benefit such as “Download Your Free Whitepaper.”

There is another place to click for visitors who already know all the information about the pallet trucks and are ready to buy. Do you see it? It takes a second, but it is at the bottom of the right column: “Yes, I want a quote for a pallet truck.” This is an important link for the actual bottom line of the company. People who click here are interested in buying. But it’s small, de-emphasized by location, and does not attract the visitor’s eye-path with color and so forth.

Someone ready for a quote does not need to download a white paper. So consider a test where this link is placed above the images, right after the paragraph. Also, a blue font will make it stand out from the other font. Blue is the Internet standard for a link and this color change will help make it more obvious that the link is clickable.

Dr. Flint McGlaughlin will next be speaking live about optimizing email response at MarketingSherpa’s Em@il Summit ’10 in Miami, Florida from January 20-22, 2010. He will also be teaching a live pre-summit Email Optimization Workshop on January 20.

Email Marketing: Building Valuable Subscriber Lists on the Cheap

December 4th, 2009

This has not been a banner year for marketing budgets by any estimation. So you might be surprised that two tactics actually garnered increased budgets in 2009 – email and social media. Your peers consider email a highly cost-effective tactic and see social media as a way to extend that content to new markets. This research comes from MarketingSherpa’s 2010 Email Marketing Benchmark Report, which contains practical data to improve your budgeting and grow your overall business.

We’ve found email marketing to be a hot topic as well, with near-record attendance at Wednesday’s web clinic (If you couldn’t attend, please subscribe to the free MarketingExperiments Journal to be notified when the replay and research brief are available). To build on that clinic, which explored ways to maximize revenue from your house list, here is a cost-effective way to grow your list:

In the past year, low-cost has become the most popular modifier of the word “campaigns” for most marketers. Of course, you never want to sacrifice results simply for the sake of cost.391609724_6a85f6981b According to the 2010 Email Marketing Benchmark Report, blog contests are an inexpensive way to quickly gain motivated subscribers. Here are the seven key steps to making the most of blog contests to rapidly grow your list:


There are highly relevant audiences for blogs on almost every interest under the sun, including Sun (Microsystems, that is) and, likely, an audience with interests very similar to your best customers. But, contrary to popular belief, these audiences aren’t all micro. According to the web-traffic analysts at Compete, some so-called “mommy blogs” get well over 100,000 unique visitors a month. For an example, see Dooce (if you’re a parent, you know what she’s referring to).

Lay down the law

Make sure you clearly define a set of rules to keep everything running smoothly. For example, you could give extra entries to readers who refer friends. Or even host a second, private contest for the blogger who generates the most entries. And remember, the more compelling the prize, the more motivated your audience will be.

…and he told two friends…and she told two friends

After you set up a landing page to explain the contest and capture entrant’s information and referrals, email referred prospects automatically and invite them to join the contest as well. With luck (and a compelling contest), you may reach the Holy Grail of cost-effective online promotion – going viral.

Seek the source

To understand which channels deliver best, create coded links to track traffic originating from blogs (with unique links for each blog), referrals, newsletter emails to current subscribers, social networks, etc. If you hold a separate blogger contest as well, you could create an anonymized tracking page to show bloggers how many entries they’ve generated compared to competitors, which may encourage them to step up efforts.

Release the hounds

Once you have the mechanics of the contest in place, finding the right bloggers will take a bit of hunting on your part. Here’s one simple strategy. Use basic Web searches to find applicable blogs. When you spot a likely target, use its “blog roll,” or links section, to find similar sites. Look at the sites’ number of RSS subscribers (if publicized) as well as the freshness of its content. Then, you can reach out to the bloggers (using info found on the site or a “Contact Us” form) with an email that includes a description of the contest, a coded link to the landing page, a link to the stats page, and a link to a promo ad.

Remember your members

While these bloggers will hopefully drive new subscribers, don’t forget to let the current members of your virtual fan club enter as well. The contest deserves at least a mention in your email newsletter, Twitter feed, Facebook group, social networks, weekly coffee klatch, Pinochle tournaments, and any other place you regularly communicate with your most loyal customers. Not only are you deepening your relationship with existing customers, making it easy for them to pass the contest on to friends is another cost-effective, viral way to grow your list.

Rinse, wash, repeat

If you do not prevent multiple signups, you will have to scrub your list of duplicates. You may also want to remind new subscribers why they are receiving your email newsletter (“Thank you for entering our contest and signing up for…”). Include an easy way to unsubscribe, a must for the CAN-SPAM Act, since some may have focused more on your prize than the fact that they were also signing up for an email newsletter. This is also a way for your least motivated list members to self-select and get removed before too many of them hit the “SPAM” button and hinder your deliverability.

After you’ve counted all your new subscribers, look at your metrics to see what you could have done better. And then, start another contest with your newfound knowledge pushing you to even greater success.

For a real-world example of a marketer that used these tactics to grow a small email list to 20 times its previous size, turn to page 129 of the 2010 Email Marketing Benchmark Report. MarketingExperiments blog readers can receive a $100 discount.

And for a more in-depth look at making email and social media deliver for your bottom line, check out Email Summit ’10 in Miami from January 20-22. PLUS, Dr. Flint McGlaughlin will teach a Pre-Summit Live Email Optimization Workshop to help you maximize your email capture rate and quality. Register by January 8 to receive an early bird discount of $200.

Photo attribution: / CC BY 2.0

Test Your Marketer’s Gut: Email frequency contest

December 2nd, 2009

Sending more than 1.2 billion emails per year is a significant marketing investment. And for one of our Research Partners, this effort raised several questions:

  • When will their list get irritated?
  • How many emails should be sent on a regular basis?
  • At what point do emails start hurting sales?

To ensure they were getting the most value from their marketing spend, our Research Partner wanted definitive, data-driven answers. So we tested for the optimal frequency that will maximize total revenue. While our scientists now have the benefit of reams of information and know the answer to these questions, we thought it would be a fun challenge to your “marketer’s gut” to test your acumen and see if you could spot a winner based on sheer intuition (and yes, there is a prize).

Background: The Research Partner is a large ecommerce company that sells well-known, inexpensive, perishable products online (if we told you any more we’d have to kill you). They had a massive, yet varying email send rate and was emailing the house list anywhere from once a week to four times a week. Most of the Research Partner’s strategy was based on the offers available at the time. With such variance in frequency, we wondered if sending more email messages would have overly negative effects on unsubscribe rates. And likewise, we wondered how much impact sending fewer emails would have on revenue. Ultimately, we were looking for that optimal email-sending sweet spot.

Test Design: We took a small, highly-motivated segment of the Research Partner’s house list and used it as our testing sample. We then split that list into seven segments that would receive different send frequencies as represented below:

    Segment 1: 1X PER MONTH
    Segment 2: 2X PER MONTH
    Segment 3: 3X PER MONTH
    Segment 4: 4X PER MONTH
    Segment 5: 6X PER MONTH
    Segment 6: 10X PER MONTH
    Segment 7: 15X PER MONTH

We monitored the effect of the send frequencies for 60 days. We tracked delivery, open rates, click-through, conversion, revenue, spam complaints, and unsubscribe rates throughout the duration test.

Email Sends GraphResults: Testing for optimal frequency assumes that revenue and unsubscribes will increase at a steady rate until the list gets irritated. At that point, revenue will experience diminishing returns and even decrease. Likewise, unsubscribe rates will increase at that point of irritation.

We wanted to test the validity of this assumption, as well as discover the optimal email frequency for this company’s email list that increased both total revenue and lifetime value of the customer.

But before we reveal the results from our scientists’ brains, we want to test your “marketing gut” with the following question (Oh, and just to spice things up a little, one person’s intuition will get them a free seat in one of our online certification courses – normally $595.):

  1. What is the optimal monthly send frequency for this company?
    1. 1-2 per month
    2. 3-5 per month
    3. 6-9 per month
    4. 10-15 per month

Congratulations to Sharon Mostyn, winner of the Email Frequency Contest, and one of only a handful of correct responses. Sharon chose the Landing Page Optimization Course as her prize. Subscribe to the MarketingExperiments Journal to be notified when the web clinic replay and research brief are available so you can see the correct answer along with a full analysis of how this discovery can help you shape your email campaigns.

To enter the contest, leave your choice as a comment to this blog post along with your email address or Twitter handle (make sure you’re following @MktgExperiments so we can reach you). We will select a winner randomly from the correct responses (and yes there is a correct answer). The winner and results for this test will be announced live on Wednesday afternoon at 4 p.m. EST during our free web clinic – Optimize your Email in Three Steps: How one marketer tripled revenue from their house list.