David Kirkpatrick

Testing and Optimization: A/B tests on landing pages, email and paid search from case studies

March 5th, 2015

No matter what type of digital marketing campaigns you are executing, there are elements in every channel that can be tested and optimized to improve campaign performance.

For example, email subject lines, copy, design and even the “from” field can be tested. Webpage elements ripe for testing include design, layout, copy, color, call-to-action button elements and more. With paid search you should be testing keywords on an ongoing basis to continually improve your PPC spend, but you can also test ad copy and calls-to-action.

At MarketingSherpa (sister company of MarketingExperiments), we publish case studies in our newsletters every week, and very often those case studies include a testing and optimization element. For today’s MarketingExperiments Blog post, I wanted to share three of those examples taken from previously published newsletter case studies.

I hope these tests give you some ideas on testing your own digital marketing channels.


Test #1. Webpage: Increasing lead generation on a landing page

This first test was actually a collaboration between researchers at MECLABS (the parent company of MarketingExperiments) and HubSpot and was conducted during Optimization Summit 2012. The full test was covered in the article, “A/B Testing: How a landing page test yielded a 6% increase in leads.”

A lead form landing page for HubSpot’s software with a free special report incentive for filling out the registration form was tested, with the Summit attendees providing input on what to test.

Before the Summit, the testing team came up with four hypothesis options:

Hypothesis 1 — Visitors arriving to the page are highly motivated to download the e-book based on brand recognition. Removing friction from the page will result in a higher conversion rate.

Hypothesis 2 — Communicating the urgency of the offer — that the free e-book download is a limited-time offer — will result in a higher conversion rate.

Hypothesis 3 — Adding more visual value to the page, such as charts and graphs from the e-book, will result in a higher conversion rate.

Hypothesis 4 — Incorporating pricing information to increase the perceived value of the e-book will result in a higher conversion rate.

The audience was allowed to choose which one to test and decided on Hypothesis 2.




Treatment (Hypothesis 2)


The only difference between the two versions was an emblem on the page, stating, “Limited Time Offer,” to add urgency to the incentive.

The test was executed during the two days of Summit. At a 97% confidence level, it determined that the Treatment outperformed the Control 6.8%.


Test #2. Email: Testing every element in an email send

International SOS is a B2B company providing medical and travel security risk services to international corporations, governments and NGOs. The MarketingSherpa Email Newsletter case study, “Email Marketing: 400% webinar attendance increase for B2B company through relevance and A/B testing,” covered how International SOS regularly tested multiple elements in its email sends to continually optimize its campaigns.

Nadia Karasawa, Assistant Marketing Manager, International SOS, explained that emailed webinar invites were sent every month to the same audience.  That consistency made it easy for the team to A/B test email elements and benchmark against previous results.

Here are some of the elements tested and what International SOS learned about its email audience:

  • Call-to-action — the team found out that repeating the call-to-action three times offered the best results
  • “Register Now” outperformed “Register” with 4% more conversions
  • An orange call-to-action button outperformed blue or gray by 5%
  • Describing the webinar as “express,” even though the 30-minute length was not changed, increased registration and attendance
  • Other discoveries include: four bullet points outperformed three, social sharing did not improve results and both six and seven form field registrations performed equally well

What was the value of constantly testing and optimizing so many email elements?

The results the team was able to achieve were:


The 2011 send vs. 2012 send:

  • A 72% increase in registration
  • A 23% increase in attendees

The 2012 send vs. 2013 send:

  • A 47% increase in registration
  • A 23% increase in attendees


Test #3. Paid Search: Testing PPC ad copy

Testing and Optimization: Effort across entire PPC funnel leads to 83,000% boost in membership application performance,” features a test on paid search ad copy, pitting four different versions of the ad against each other. This test was run by GS1 US, a nonprofit that issues prefixes used to create U.P.C. and supply chain barcodes.

The entire optimization program covered every element of its paid search, from ad copy to landing pages and finally the application form that served as the final conversion of the PPC campaign. For this post we are focusing on the ad copy test.

The team created four versions of the PPC ad and compared results:


Ad #1:

Need to Get UPC Barcodes?
GS1 US, the only authorized source
in the U.S. for your U.P.C. barcodes

Ad #2:

Need to Get U.P.C. Barcodes?
Buy Authentic U.P.C. Barcodes
only from GS1 US Apply Online

Ad #3:

UPC Barcodes
GS1 US, the only authorized source
in the U.S. for your U.P.C. barcodes

Ad #4:

UPC Barcodes
Buy authentic U.P.C. Barcodes
only from GS1 US Apply Online



Ad #1 won the overall test with the highest clickthrough rate, besting Ad #2 by 110%, Ad #3 150.7% and Ad #4 by 252%, all at a 99.7% confidence level for the test.

One reason GS1 US tested two sets of ad titles and body text was to find the optimal combination. However, the tests were also designed to discover which language best resonated with its target audience to use for messaging in other venues, such as landing pages.


David is a Reporter for MECLABS Institute. You can follow on Twitter at @DavidOnline.


You might also like

Online Testing: Why are you really testing? [More from the blogs]

Email Testing: How the Obama campaign generated approximately $500 million in donations from email marketing [MarketingSherpa case study]

Email Marketing: Education group utilizes A/B testing to increase open rates by 39% [MarketingSherpa case study]

Email Preheaders Tested: The surprising sensitivity of a single line of text [More from the blogs]


Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Analytics & Testing Tags: , , , , ,

Selena Blue

Email Marketing: Top 5 most effective list growth tactics

March 2nd, 2015

In a digital world, where 72% of U.S. adults prefer communication with companies through email, how do you capture new email addresses?

Acquiring emails for our list is a continuous task. We have to work to not only retain the list we have, but to also grow it to build a larger audience of prospects and customers.

In fact, 63% of marketers reported “growing and retaining subscribers on our list” as a marketing goal, according to the MarketingSherpa Email Marketing Benchmark Report.

But with so many tactics and strategies out there, where do you begin?

In the Benchmark Report, we asked email marketers several questions about the different tactics out there. This MarketingSherpa blog post will break down the five most effective list building tactics, as reported by your peers.


Tactic #1. Registration during purchase

The most effective tactic according to respondents was gaining email addresses during the checkout process. This make perfect sense. After all, you already need an email address from customers during the online purchasing process. You’re not technically asking for anything they aren’t already giving.

It simply requires adding a small checkbox for customers to check if they’d like more information, promotions or discounts. This could be why 52% of marketers said it was also a very or somewhat easy tactic to implement.

However, I caution you to think about how you implement this. You don’t want to prevent customers from purchasing because of a confusing or required registration or list sign-up.

To learn about the two registration options — front-end vs. back-end — read the MarketingSherpa Blog post, “E-commerce: Why a forced checkout registration is never a good idea.”


Tactic #2. Website registration page

The second tactic is viewed as the easiest, with 85% of respondents saying it’s very or somewhat easy.

There are many ways to add a registration form to your site, whether it’s a form on the homepage or a landing page all its own.

Even if you already employ this tactic, it could be worth it to reassess your current strategy. Kodak revamped its strategy, including updating its capture page and adding more opt-in requests, to increase email subscribers by 33%.

According to the Benchmark Report, 29% of marketers found white papers and other premium content is effective for registering new email subscribers on their sites. This could be for one-off downloads or to access a library of premium content. To see how a free paywall can grow your list by using content you already produce, see how Copyblogger grew its email list by 400%.


Tactic #3. Online events

Online events, or webinars, are no exception to the rule that providing prospects with content is a great way to achieve a value exchange. Customers get valuable information in exchange for their email address.

Webinars can require a significant amount of time, planning and resources, so it’s not a surprise that only 31% of marketers say they are very or somewhat easy to execute. However, as the third most effective tactic — with 37% saying it’s very effective — they might worth the investment.

Partnering with another company could be a great strategy for webinars. One, you have another set of hands to help with the webinar creation and execution. Two, you’ll have access to another email list, potentially filled with new customers.

You can create buzz around the event through social media, blog posts and even paid advertising. HubSpot attracted 25,000 sign-ups, which turned into 10,000 attendees. The team was then able to turn 3,500 of those attendees into new leads. Learn how the team achieved these results in the MarketingSherpa case study.


Tactic #4. Offline events

Depending on your industry, you might be involved in different offline events. Trade shows, for both B2C and B2B, can be a great place to build awareness within your market. It’s also a great opportunity to gain email addresses.

A popular tactic is a drawing for a free prize, where entrants must hand over their email address. However, there are other ways to stand out on the show floor.

Loggly — a cloud log management company — created an event sponsor guide for an industry event it was sponsoring. Reaching out to the other sponsored vendors, the team created a crowdsourced guide they posted on their blog.

The company generated its highest level of engagement with the event, and while they posted the guide to a blog post, it could have also been easily gated behind an email capture page. Learn more by reading the case study: “Tradeshow Marketing: Why a SaaS company helped competitors generate leads.”

A great offline tactic for brick-and-mortar retail stores is offering discounts and promotions in exchange for an email address. KooKoo Bear Kids, a children’s furniture and décor retailer, set up iPads for email list sign-ups in plain view of the checkout counters. The retailer found three benefits to this method: 

  • Customers use a popular device, which adds to the appeal of signing up.
  • Customers do not have to say or spell embarrassing email addresses they may have created years ago.
  • KooKoo Bear does not have to manually enter email addresses at the register or from a slip of paper.

This tactic, along with asking for sign-up during online checkout and on most pages of the website, has increased list growth by 15%. Read on to learn more: “Email Marketing: 142% higher open rate, 15% bigger list from retailer’s strategy.”


Tactic #5. Paid search

While a quarter of marketers thought paid search was a very effective way to gain new email subscribers, not many more — 29% survey respondents — reported actually using it to drive list growth. This could be because 80% of marketers reported paid search as somewhat or very difficult to use.

You could use paid search to increase traffic to your site overall or to draw traffic to a sign-up page for a webinar or landing page for a white paper.

While the case study, “PPC Marketing: Testing value proposition messaging increases clickthrough 88%,” uses paid search to increase site traffic, not list growth, it does detail the process of finding the best PPC ads for your potential customers.

Understanding how value proposition affects the success of your ads could not only increase the effectiveness of your campaigns, but it could also reduce the difficulty of knowing what to focus your ads on.


But above all else, test to know what is most effective for your audience

Just because one tactic is effective for many companies doesn’t mean it will be best for your customers. After all, if you don’t have an online purchase option on your site, adding an opt-in during checkout won’t help you.

This list just provides you a starting point for optimizing your email strategy. To know what draws in subscribers to your company, you’ll want to test multiple tactics.

Take Techlicious, a site that publishes simple and straightforward advice on technology for women, as an example of the power of testing when it comes to email list growth. Over five months, they tested six channels to determine which generated the greatest number of engaged subscribers. Read the case study to learn how they garnered 230% more readers from the six-channel test.


You might also like

Email Lists: How sweepstakes work for CNET [Live from #SherpaEmail] [More from the blogs]

Lead Generation: How to build your own list [More from the blogs]

Marketing Research Chart: 63% say registration during purchase effective for list building [MarketingSherpa Chart of the Week]

Lead Generation: Content and email combine for high-quality list building [Case study]

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Email Marketing Tags: , ,

Erin Hogg

Email Lists: How sweepstakes work for CNET [Live from #SherpaEmail]

February 26th, 2015

Before jetting out to MarketingSherpa Email Summit 2015, Daniel Burstein, Director of Editorial Content, MECLABS Institute, and I took a little trip down memory lane and reviewed some top takeaways from Email Summit 2006.

Of course, it’s exciting to see how some things have changed and laugh at how far we’ve come. In 2006, consumer marketers were warned about Yahoo and MSN Hotmail adding preview panes to email services.

We were using words like “ezines” and “hotlinks.” We assumed email would be dead because of junk mail. We were just getting the hang of using Web analytics and email systems together to track customer value.

While we have come a long way since then, there were some things that have stood the test of time.

Keeping opt-ins actively engaged with email content is key to improving ROI. Don’t have Marketing operate in a silo — work in coordination with not just Sales but also IT to gain the right solutions and tools you need to succeed.

One takeaway caught our eye, as it is something I discussed onstage with Diana Primeau, Director of Member Services, CNET, earlier this week at #SherpaEmail.

In 2006, we heard from David Kreitzer, then Marketing Director, Bella Pictures, and his advice for using sweepstakes to build email lists.

According to David, although sweeps and free bonus offers can dramatically raise email opt-ins, list quality suffers. You may get tons of new names on a list, but they could just be there for the contest.

I’m sure many marketers even now have been advised not to use sweepstakes or contests to build a list.

Fast forward to yesterday, Diana shared how one way CNET builds its list is through sweepstakes and had the numbers to prove that subscribers can stay actively engaged post-contest.


What do we know now that’s different from what we believed in 2006?

It’s simple really: create a relevant experience with users with personalized content to retain them for the long term.

We knew this back then, but our capabilities over the years have grown to allow marketers to leverage the tools and platforms they need to segment, test and optimize an email experience for each subscriber, serving up content that creates a one-to-one relationship.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Email Marketing Tags: , , , , , ,

Taylor Kennedy

Live from Email Summit: Two tactics to reduce perceived cost in your email capture forms

February 23rd, 2015

I’m reporting live today from the MarketingSherpa Email Summit in Las Vegas, one of the most exciting weeks of the year for email marketing practitioners (woo hoo!). While I never imagined being this pumped up about email marketing growing up (I had a passion for dinosaurs and Transformers as a kid), it’s incredible to see this many email marketers in one place sharing what works for the benefit of the whole industry.

Today marks the start of the Summit with a workshop on “Effective Email Messaging” taught by Flint McGlaughlin, Managing Director and CEO, MECLABS Institute, and the MECLABS team.

One of the concepts that seems to be resonating especially well with the marketers in attendance today is the concept of email marketing as a continuum, where the relationship you establish in the email capture form (lead generation form) affects each interaction that occurs thereafter.

Therefore, starting your “email relationship” off on the right foot with a well-thought-out email capture strategy is of critical importance. Let’s avoid any and all applicable first date metaphors and dive right into the key principles that Flint has covered in this session, backed by tests to support them.


A framework for reducing perceived cost of the email capture

When testing your email capture fields, where do you begin? It’s important to remember that any action you wish the visitor to take on your website is a balance between two forces — cost and value. In order to increase the likelihood of the visitor taking the desired action, you should always be seeking to minimize cost and increase value.

In today’s post, we’ll be exploring the cost side of the equation.


Typically, one of the easiest places to start is by looking at your email capture forms for any unnecessary elements that might contribute to the perceived cost.

Cost takes on two forms when dealing with email:

1. Amount of information required — Think the amount of form fields involved. How many form fields are actually getting put to use by either the marketing or sales team once captured. If you’re not using  a particular form field currently (or not planning on using it in the near future), then get rid of it. Also, make sure your audience is clear on which form fields are required to participate in the email list.

2. Nature of the information required — This is the type of information that is required in the form fields. If you require a telephone number for a digital download, this might convince the visitor that you have an ulterior motive for their contact information. Also, think about more personal information types, such as driver’s license number or social security number. This information may be more difficult to obtain or could cause concern to the visitor.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Email Marketing Tags: , , , , ,

John Tackett

Landing Page Optimization: How a navigation test increased leads 34%

February 19th, 2015

Site navigation can make or break the user experience on your site.

So what can you do about it?

In today’s MarketingExperiments Blog post, I wanted to share with you an interesting experiment from our latest Web clinic that shows how a tourism group increased leads 34% by testing their site navigation.

Before we get any further, let’s take a look at the experiment:

Background: A tourism commission seeking to enhance visitor interaction with their website content in order to boost appeal for choosing their city over other destinations.

Goal: Test the site navigation to increase visitor engagement with key site content.

Primary Research Question: Which navigation type will increase site engagement?

Secondary Research Question: Which navigation type will lead to the highest lead generation rate?

Test Design: A/B/C split test

In the control above, the MECLABS research team hypothesized that the navigation was increasing user friction by including an almost-overwhelming amount of options.

Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Landing Page Optimization Tags: , , ,

Sam Caruso

SEO Marketing: Adding value without risking search rank

February 16th, 2015

It’s common knowledge that search engine optimization (SEO) often plays a major role in how companies group their key terms, whether they be in the headline or in the bottom of a page.

This practice can also lead some companies to avoid testing certain areas of their site in order to maintain page rank.

However, there is one area where company value can easily utilized without risking online ranking. Here’s a recent experiment where we tested one minor change in an SEO headline and achieved a significant lift in conversion.

The MECLABS’ Research Partner wanted to concentrate on headline testing for one of their high-ranking SEO pages but had a few stipulations on how the headline copy could be laid out.

To avoid any ranking pitfalls, we went over various value points for the Research Partner to find the best way to incorporate value without damaging any SEO rankings if any treatment were to outperform the control (the existing high-ranking page.)

After careful review of the partner’s various value points, we found that the partner had a price guarantee that presented good value to potential customers.

For the experiment, the copy for the treatments had to be worded carefully as to not interfere with key search terms. To keep in line with the partner’s SEO parameters, we developed logos depicting the price guarantee alongside headlines that featured the word “guaranteed” and mixed them with the key search terms used in the headline.

The main rule that had to be followed when putting the treatments together was that the first two words of the existing headline had to remain where they were to keep search ranking in place. The following logos were placed next to the headline to express company value:

  Read more…

Share and Enjoy:
  • LinkedIn
  • StumbleUpon
  • Facebook
  • del.icio.us
  • Digg

Categories: Search Engine Optimization (SEO) Tags: , , ,