MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

Email design always proves to be a hot topic with marketers. And when you have top agencies competing against each other, the fire just gets hotter as we learned during last week’s live web clinic – Maximize your Agency ROI: How adding science to the creative process reveals a 26% gain.

We received a plethora of questions, most which we could did not have time to address during the hour-long clinic. So, as with every Web Clinic Extra, we have picked a handful of the most common questions to address here on our blog. This week we pulled in Andy Mott, the Senior Manager of Research Partnerships, to answer these questions…

Email marketing is a topic that comes up often in the MarketingExperiments community. In fact, Dr. Flint McGlaughlin is delivering a keynote today at Em@il Summit ’10 in Miami as well as teaching a pre-summit Live Email Optimization Workshop. If you couldn’t make it out there this year to get valuable insights from your peers and industry leaders, come back to the blog on Friday for some key takeaways from this year’s summit.

You can view a replay of the clinic or read the latest issue of MarketingExperiments Journal. Our next live web clinic, The Five Best Ways to Optimize Email Response (Part 2): How to craft effective email messages that drive your customers to action, will be taught on February 3rd from 4 to 5 p.m. EST.

Clinic Notes, Email Marketing, Marketing Q&A

Students and alumni of the MarketingExperiments Training and Certification Program often share their questions and concerns with our analysts before, during, and after they take one of our courses. The questions below are an example of the interaction you can expect if you attend a MarketingExperiments course:

Q: Do you know any other resources, except MarketingExperiments and MarketingSherpa, that are good at providing useful insights from case studies?

Believe it or not, I’ve been in this Internet marketing field for two years (you’re probably here a lot longer) and been through a lot of misleading information until I accidently found you guys and really learned how to test things and see if they REALLY work instead of blindly believing some “gurus” who told me something like…”this is tested and it’s working 100%!” (the only thing that was truthful was that 100% thing…the problem was that almost 100% of what they were telling me didn’t work.

Surely others – for example online marketing consultancies – will offer to advise you on changing your website to improve performance and will use a third-party testing tool to measure the impact. Also, some of the large-scale testing tool vendors offer hosted and/or managed service engagements using their products.

Unfortunately, as you said, most other organizations are not research focused. MarketingExperiments is a research institute dedicated to discovering what really works in online marketing to help our Research Partners, certification program students, and Journal subscribers succeed. So there are very few resources we can point you to.

One notable exception is the award-winning Get Elastic blog written by ecommerce analyst Linda Bustos. Get Elastic provides useful insights about SEO (search engine optimization), usability, analytics, email, shopping cart abandonment, and social media. Linda is also a MarketingExperiments certified optimization professional and knows our methodology inside-out.

Q: I’ve been through your Fundamentals of Online Testing course. You teach about landing page and order page optimization. I was wondering if you have some specific advice/studies where the SEO effectiveness of a campaign was tested (SEO, not paid traffic) because I can’t find any valid way to measure the effectiveness of an SEO campaign.

Regarding testing in which the primary channel is “natural search” or SEO traffic, we do have considerable experience working with companies and organizations for whom SEO is a significant portion of their demand, and we have published some research on the topic. In fact, all of our research is readily available for free in the MarketingExperiments Research Directory.

As you’re already aware, based upon your question, there are a number of challenges associated with the dramatic differences between the key optimization factors over online marketers can control when choosing between PPC (pay per click) and SEO. Specifically, not only is there a relative dearth of information available to search marketers as compared to paid advertising, the search networks are comparatively opaque about their results-positioning algorithms and tend to change them frequently to confound SEO-gamers. Further, the rate at which changes to a site are detected by the networks and “shaken into the mix” is volatile and unpredictable.

Consequently, the MarketingExperiments approach is to evaluate the subject site/page based upon the principles of Offer/Response-Optimization – such as the Conversion Heuristic that you learn in the Landing Page Optimization course – then develop hypotheses about how to improve its performance and test those hypotheses using paid search traffic (which is designed to mirror the motivation profile of their ideal customers through SEO). This provides comparatively rapid and precise evidence about the specific factors of study. Then, those test results are used in concert with the latest SEO-algorithm information to develop the channel-specific page designs and a plan for deploying them to get the largest and most rapid performance gain.

Q: Do you have some advice/case studies about testing the effectiveness of an online service?

For example, a site like Traffic Bug submits your URL to social networks (Connotea, Propeller) automatically and claims that this increases your rankings and indexes your pages.

I want to do a test to see if what they say is true. I would take 10 very obscure pages (to be exact, profile links) that are rarely indexed by Google. I create 10 of those obscure pages on different URLs and do nothing with them. Then I create 10 pages on the same URL and submit them to Traffic Bug. I wait for seven days.

Of course, I make sure everything is satisfied in terms of validity and all that. So I wait for seven days (that’s the first milestone, I then check them again after 30 days but let’s focus on the period after seven days) and then take a look. My sample size is small but what I’m interested in is discovering if this service is highly effective in indexing pages on Google.

So if the first sample (that is not submitted) gets one out of 10 pages indexed and the second sample (which is submitted) gets nine out of 10 pages indexed, and I make sure that this is a valid test (using the MarketingExperiments validation formula from the Fundamentals of Online Testing course), can I assume an online service is very effective?

I wanted to hear your comments on this. What do you think is wrong/right with the above test and what would be some things to do for improvement? Also, do you think that a sample size of 10 is big enough for a test to discover whether an online service has a dramatic effect?

While the approach you described appears sound in principle, you will probably have difficulty actually achieving valid results based upon the circumstances you outlined. And even if the results are valid, they may not really answer your question.

In evaluation of a tool like this, a different approach may serve you better. When building an SEO campaign and links there are other things to consider:

1. Are you sure where all these links are getting posted? Some indexing tools use less-than-kosher link-building strategies that can actually get your domain in trouble with search engine providers. The appearance of link spamming and posting links on flagged sites can cause domains to suffer penalties that can affect the ranking of their sites…occasionally on a permanent basis.
2. Are these links actually driving traffic and revenue? Many indexing services cost money and you need to perform due diligence with an ROI analysis to see if the efforts are recouping their costs.
3. Does the service provide a list of links they have generated for tracking? Not only is this good for tracking but allows you to see the places your links are getting placed. Some business owners consider it (as you should too), important to see the company you are keeping on these sites.  For example, are links to, “Adam’s XXX site” right next to your link or the content on these pages? You can use tools like Yahoo Site Explorer or Google Webmaster Tools to fish out these links, but the service should do this for you.
4. It is important to note that we are not accusing Traffic Bug of doing any of these things, but with any sort of service along these lines you need to do your research first.

From your experience, how would you answer the above questions? Share your advice in the comments section.

Special thanks to Director of Sciences Bob Kemper and Research Analyst Corey Trent for their help in answering these questions.

Marketing Q&A, Practical Application, Research Topics, Search Engine Optimization (SEO)

You can hear Senior Manager of Research Partnerships Andy Mott answer the question How Can You Make Your Web Site Smarter? on the replay of Omniture’s latest webinar. But in my experience with these events, there is usually an interesting back story. So I cornered Andy in his office at a vulnerable time (his beloved Gators had recently lost the SEC Championship game) and found out what he really wanted to say…

Q: You discussed the 2009 Omniture Online Conversion Survey on a recent Omniture webinar. What surprised you the most?

Well I won’t say this surprised me. Maybe saddened is a better word. The survey asked “How frequently is online marketing testing employed in your company?” About half of the respondents said infrequently or never.

Q: Wow! That is pretty shocking, especially considering that these people are already familiar with testing through Omniture or MarketingExperiments. Maybe I could understand if this were the general population of marketers. But why have testing tools in place and not test? Why do you think half of them are flying blind?

Well people know they need to test. They probably know their competitors are testing and getting results. But the idea of executing a test is such a paradigm shift in the way that they’ve always done things.

Those that are higher in an organization tend to be more experienced. And if they are more experienced, they may be locked into the advertising agency way of doing things from 30 years ago, just like the doctor who overlooks recent findings and does what worked best for him when he went to medical school.

Q: Change is difficult. But still, thirty years ago these same people were also wearing polyester and doing the hustle. I’m a little skeptical that they would still try to shoehorn old media principles into new media.

It’s not intentional. If something has always worked for you, why change?

But what we really have is the tyranny of best practices. I’ll give you a great example. Many marketers still believe that they must have the call to action “above the fold” on a web page. Yet testing has shown this to be an utter myth.

Q: And nothing disproves a best practice better than a test that shows what actually works for their specific situation.

That’s the thing. Once companies start testing and see the ROI they are absolutely hooked.

Q: How do you take that first step? For, say, an email marketing manager reading this, how do you create a culture of testing in an organization?

Business-level executives don’t care about optimization or testing or even online marketing really. What they care about is results. So you need to talk to them in their language.

At MarketingExperiments, we publish all of our research and it is available for free. So go to the research archive and pull some experiments so you can show example results and make the business case for testing. At this point, all you are looking for is a small budget to begin testing.

Those first tests will help you establish a beachhead that you can use to further penetrate the organization. Because once businesses see the results they can gain from testing, it can get addictive. It’s like eating chips while watching a football game, you just can’t stop.

Q: The challenge is to just get the ball rolling. This sounds great in theory. Do you have any real-world examples?

I have countless examples. Since we started this conversion by talking about my recent webinar with Omniture, let me tell you about a Research Partner that first got interested in testing by attending an Omniture webinar that featured Dr. Flint McGlaughlin.

Companies that test usually like to stay anonymous because they view this process as such a competitive advantage. So I’ll just say they are a very large financial institution.

So this marketing manager attended Flint’s webinar and was totally sold. He was convinced that they should begin testing. But he’s only responsible for a very small patch in this giant company. It took him six months to get the approval to begin testing, doing the things I previously mentioned.

Q: Six months? It’s easy to get discouraged in that time. I’m not sure how many people would see it through.

But here’s the kicker. That marketing manager and his boss are now charged with trumpeting this win across the entire organization. He is now in front of his boss’s boss’s boss presenting his test results. In fact, in a few days he will be presenting in front of the SVP committee that advises the CEO.

Q: Well then he must have achieved some really out-of-this-world test results. What did he get…three digits…four digits? I mean, how common is that?

They got a 38% revenue boost over what the agency was doing.

Q: Well, that sounds decent, but a committee of SVPs really cares about 38% in one test?

You say that because you are so used to the power of testing, so you just want to see huge numbers. Let me put this another way – by not testing they would have been leaving 38% more money on the table since the cost of testing was infinitesimal compared to their massive marketing budget.

And that’s the thing. This company has a huge marketing budget. They sponsor the Olympics. They name stadiums. They purchase a ton of media. And since they don’t have space to sell in most of these executions, they’re driving everyone to the website. So if they find they could make more 38% more money without having to increase any of these huge marketing spends, the increase in ROI is humongous. Even a one or two percent increase could make or break a quarter.

Q: I see. I didn’t make the connection to that old media marketing spend. But I would think it goes beyond just old media driving people to a website. Online marketing is growing by leaps and bounds. I would think companies want to make sure they are getting a return on that investment as well. According to Forrester Research, digital spending will nearly double over the next five years at the expense of traditional marketing.

Forget five years from now, even today companies spend more than $25 billion on interactive marketing – things like mobile marketing, social media, email marketing, display advertising, and search marketing. That is 12% of all advertising spending. So when enterprises, like that financial institution I discussed, learn that they can take just a tiny fraction of the spend on this growing segment and invest it in a way that ensures the effectiveness of everything else they do – with real-world, statistically valid data – they get very excited.

Q: And I would think, for the employees that can tell management “I know how to get the best ROI from this” – not think or have an opinion, but know with real numbers – that’s quite a smart career move.

If other people are discussing so-called “best practices” and you’re showing real results, then you become the go-to person. The one who knows how this stuff really works. Because nothing defeats the tyranny of best practices as well as the audacity of testing.

And if you’re the guy that knows the right things to do in an explosively growing field like Internet marketing, while marketing budgets on everything else are falling, you’re in a good place no matter what the economy is doing.

How did you get the ball rolling on testing in your organization? What are your biggest challenges to create a culture of testing in your organization? Share your triumphs and challenges in the comments section below or post them to our MarketingExperiments Optimization group.

Internet Marketing News, Internet Marketing Strategy, Marketing Insights, Marketing Q&A, Practical Application

During the Optimize your Email in Three Steps web clinic, I covered several measurement strategies to help you measure and prove the real value of your email campaigns. I was inundated with questions. Marketers are constantly in search of new “tricks” to find the perfect numbers that help them understand and tell the real story of their Internet marketing efforts.

While I was able to answer a few of these questions on Web Clinic Extra, I wanted to dive a little deeper today with some links and walkthroughs showing how to implement some of the metric items discussed. And please note, while these examples use Google Analytics, Omniture and many other companies have excellent tools with similar capabilities.

Tagging links within emails so you can measure email clicks within your Google Analytics

Requirements:

• Links tagged in email with Google Analytics tracking variables
• Destination Pages from the email with Google Analytics tracking code installed

Walkthrough:
First, with your emails, identify what links you want to track. For some people, just tracking CTA is enough, for others looking at additional navigational links (for example a supplied news article link or a support link) is also valuable data as well. Once you have compiled a list of links that you want to track, visit Google’s URL Builder Tool and start building your links. Please note that campaign source, medium, and name are minimum input requirements for this sort of tracking to work. You also have some remaining variables (name and content) you can use to insert segmentation data. In the example below, you will note that we inputted some demographic and business data:

Once you have built your links, insert them in the appropriate places in your email and hit the red button.

Please note, that using this tool is not necessary to build these links. Once you learn what variables are used, you can build a script that will automate this for you. You can then use internal databases of customer information to create dynamic and automated email tracking.

Also, once these emails go out, you can then create segments on these parameters and get targeted and segmented metrics for your email efforts:

As a final note, make sure you install Google Analytics on the page your audience will visit. This will be required to measure the clicks. Google Analytics tracking code is not required to be in the email, just the landing or website page they are landing on. The tracking script will read the URL variables that you put in your links in the email and recognize the data.

You can also apply these metrics to ecommerce and other reporting data within Google Analytics, giving you a further layer that attributes efforts to the bottom line.

How to incorporate form fields in goal reporting

Requirements:

• Adding the “onClick” markup JavaScript function in the form field you want to track
• Page must have Google Analytics tracking code installed

Walkthrough:
When I reviewed an example goal setup in the Optimize your Email in Three Steps web clinic, one of the steps I mentioned was a form field click as a goal step. In reviewing the clinic comments, I was stunned by the number of people that wanted to know how to do this and for me to explain further, so here we go.

First, as part of looking at email performance, many of us are sending users to pages that have form captures. For me, a great user experience or path to look at is users that click from the email, land on the target page, actually click into the form, and then submit/convert. So let’s look at a typical form code example, and how Google Analytics (GA) ties in:

Standard form input code example:

We can insert an onClick function to the form to capture when a user clicks into the field and complete the information. With this function we will be making a call to the GA tracking function: _trackPageview. What this function will do in our case is when a user clicks into the form field a page will be created in Google Analytics that we specify/create. For example purposes, with the page tracker function we will create the page /dec-email/form-field-email1.html.

After users have interacted with the form field, the /dec-email/form-field-email1.html will start to appear. Just to clarify, this page does not exist, but we have told GA to record clicks and interactions to the tagged form field to this mythological page we have made up. Also, if you are doing email testing, you could create a script that recognizes which email people are coming (e.g. URL variable) from and change this page dynamically as well. So instead of posting clicks to /dec-email/form-field-email1.html page, we use email2.html. Here is an example of Google Analytics markup on the form field:

Also, users tend to be quite “click happy” on forms, so make sure you look at unique visit data on these “pages,” rather than pageviews. Pageviews tend to be inflated because of this user behavior.

Lastly, once these “pages” are created in Google Analytics, you can insert them in goal funnels, just like other real pages. Your metrics will not skip a beat. Here is an example goal funnel that you could create in Google Analytics with the items we have covered:

**Make sure, if applicable, that you select the required step in your goal setup.

Leave a comment below and let me know the next measurement tricks you would like me to pull back the curtains on in future installments of the Magical Metrics Tour. Also, let me know if you find posts about custom or deeper metrics helpful.

For a more in-depth look at making email and social media deliver for your bottom line, check out Email Summit ‘10 in Miami from January 20-22. PLUS, Dr. Flint McGlaughlin will teach a Pre-Summit Live Email Optimization Workshop to help you maximize your email capture rate and quality. Register by January 8 to receive an early bird discount of $200.

Analytics & Testing, Clinic Notes, Email Marketing, Internet Marketing Strategy, Marketing Q&A, Research Topics

During our December 2 web clinic, Optimize your Email in Three Steps, Boris Grinkot, Heather Andruk, and Corey Trent answered questions from our audience about email relevance, frequency, and metrics.

We often don’t have time to answer all of our audience questions on the live web clinics. So we distilled all the questions into a few representative queries, and pulled Heath Andruk and Corey Trent in from the lab to share their insights on the latest edition of Web Clinic Extra:

Corey and Heather answered these questions:

Question 1 (1:05): In the frequency experiment shown on the clinic, was there variation in the types of emails (i.e. reminders, offers) in the low frequency email group (i.e. 1-4 per month) like there were in the higher frequency group?

Question 2 (1:45): What are some factors to determine good segments?

Question 3 (2:40): What do you do if you cannot segment?

Question 4 (5:00): Is there one age group more tolerable to frequent emails than others?

Question 5 (6:40): How do you figure out the best timing for emails?

Question 6 (9:55): Can you track goal pages that are outside of your domain with Google Analytics?

Question 7 (10:55): Can you track an email campaign in Google Analytics if you are sending emails with a 3rd party provider?

Come back to the blog on Friday for a technical addendum from Corey Trent. He has some specific tips to help you put his metrics wizardry into revenue-generating practice for your email campaigns.

The complete Flash version of the web clinic, along with a downloadable research brief (PDF), are now available on MarketingExperiments.com. If you have additional questions, use the comments section below or post them to our MarketingExperiments Optimization group.

Clinic Notes, Email Marketing, General, Marketing Q&A, Research Topics

During our November 11 web clinic, Surprising Wins from 2009: Using insights from an uncertain economy to drive 302% growth, Boris Grinkot, Adam Lapp, and Paul Clowe answered questions from our audience on a range of topics pertinent to our research from this year.

We often don’t have time to answer all the questions from our international audience on the live web clinics. So we’re launching a new feature on the blog – Web Clinic Extra. We distilled the best questions and posed them to Researchers Boris Grinkot and Adam Lapp for our first episode:

The complete Flash version of the web clinic, along with a downloadable research brief (PDF), are now available on MarketingExperiments.com. If you have additional questions, use the comments section below or post them to our MarketingExperiments Optimization group.

Clinic Notes, Marketing Q&A