Archive

Posts Tagged ‘a/b testing’

The Importance of Customer Centricity in Evolving A/B Testing

June 29th, 2015 2 comments

When we talk about A/B testing, we often think about it on a test-by-test basis. While this singular focus is beneficial, it overlooks testing’s role as an ongoing system for optimization.

At MarketingSherpa Email Summit 2015, Courtney Eckerle, Manager of Editorial Content, MarketingSherpa, sat down with Lauren Wagner, Senior Manager, and Tessa Srebro, Lead Generation and Marketing Associate, both of VolunteerMatch, to discuss how to utilize A/B testing to constantly evolve programs.

Lauren and Tessa’s company, VolunteerMatch, is a nonprofit organization that is the world’s largest volunteering network. It also has an interesting funding model. The company sells a Software as a Service (SaaS) product to companies to help fund its organization and, so far, this model has generated almost $1 billion in social value each year through its work with nonprofits and volunteers. VolunteerMatch is also more than familiar with the benefits of testing, as evidenced by the subject line test it ran with MarketingExperiments.

Watch the excerpt below from the MarketingSherpa Media Center to learn how keeping up with current trends and thinking like a consumer can help evolve your testing and your company.

 

Keep up with current trends and your customers

Keeping up with current customer trends is one of the most important pieces of advice Lauren Wagner gave when asked how to keep a company’s A/B testing constantly evolving. Be sure that you’re keeping up with current trends in the marketplace, but don’t forget to take the time to learn about your customers.

Read more…

Does Brand Help or Hinder Conversion? The answer is “yes”

June 18th, 2015 2 comments

Does brand truly drive revenue for every company — even if it’s not Google, Nike or Apple — or is it just a marketing department’s raison d’ê·tre? Is it really as powerful as marketers think it is?

The latest MarketingExperiments Web clinic answers this question with (of course) testing. Be sure to watch it to learn why it’s critical to be thoughtful about how and when you use branding elements.

Consider this anonymized test:

Background: A regional marketing commission.

Goal: To raise awareness of local activities and events, increasing number of travelers and tourist spending in [City].

Research Question: Which sign-up page will generate the most responses?

Test Design: A/B split test

 

The Plain and Simple Control

The Control was a landing page that encouraged people to sign up for a free guide.

 

The New and Improved Treatment

The Treatment added details — font, images and design — that better reflected the brand.

Read more…

Landing Page Optimization: An overview of how one site increased leads by 155%

June 15th, 2015 4 comments

Simple, direct and bare. When your company and process is known around the world, a blank page with little competing content can not only work, but it can work really well.

Simplicity is key. Take a look at Google’s homepage:

 

What about new visitors? Imagine coming to this page for the first time, with little to no context of the company. What is this company? If I type something in that text box, for example, where will it take me?

Simplicity is not always a key to effective website optimization.

Leaders must grow comfortable with paradox and nuance. Clarity does not equate with simplicity.  Simplicity does not equate with easy.” — Flint McGlaughlin, On the Difference between Clarity and Simplicity.

Simplicity is the reduction of friction, but clarity is the optimization of the message. A simple message is not necessarily a clear message.

Take a look at a test we ran with a physicians-only social network that allows pharmaceutical companies to conduct survey research and promote products to their audience. The goal of this A/B split test was to identify which microsite would generate the most total leads.

Check out the control below. Can you find the value proposition?

Read more…

The Power of a Specific Offer to a Specific Prospect

May 7th, 2015 No comments

Specificity converts. In marketing, there should be no such thing as a general message. The marketer communicates with an aim. This aim should dictate everything else we say. This aim should influence, even constrain, every word we say.

— Flint McGlaughlin, Managing Director and CEO, MECLABS Institute

Specificity converts. A specific offer to a specific person will outperform a general offer to a general person.

This concept relates to a recent email test we ran with our MarketingSherpa audience and ran again (with a slight twist) with our MarketingExperiments audience.

First, in case you’re not familiar with MarketingSherpa, allow me to briefly explain our sister company.

MarketingSherpa’s content is geared toward exploring general marketing principles. This is also where companies and marketers can share specific marketing stories, such as Mellow Mushroom’s social media strategy and Red Bull’s content marketing advice.

Alternatively, the MarketingExperiments audience delves more specifically into the tactics of marketing strategy. MarketingExperiments is about specific tests that the reader can apply to their own marketing.

Now that you understand the difference in content related to the tested audiences, let’s get into the test itself.

 

The test

We tested an email invitation for a recent follow-up Web clinic MarketingSherpa hosted. The clinic’s objectives were to examine the results of a live optimized email test, which was run by the MarketingSherpa audience at Email Summit 2015 alongside Flint McGlaughlin.

The test consisted of two treatments:

  1. Treatment A focused on the Email Summit follow-up test, only mentioning live optimization from the MECLABS team.
  2. Treatment B switched the emphasis by focusing on the live optimization from the MECLABS team, only mentioning the Email Summit follow-up test.

In essence, both emails were an invite to the same Web clinic and messaged the same two offers, just with different expressions of focus.

 

Treatment A: Email Summit follow-up

Subject line: Does peer review work? See the results of the audience optimized email from the Email Summit 2015

Preheader: Plus live email optimization from the MECLABS research team. 

Read more…

Here’s Why Most A/B Testing is Boring and Not Really Worth the Trouble

April 6th, 2015 1 comment

Do a quick Google search on “things to a/b test on a website,” scan the results for a moment, then come back and read the rest of this article.

Most of you reading this are marketers, so you know I’m taking a big risk by telling you to go do something else before you read my article.

In fact if you’re reading this now, you’re probably one of the very few who made it back from that incredibly distracting activity I had you do. Thank you. You are exactly the person I want to be reading this. The others can go on their merry way. They are not the ones who need to hear this.

I had you do that search because the Internet is full of people telling you to test things on your website such as color, button size, layouts, forms, etc. I wanted you to get an idea for what’s out there.

Now, I want you to understand why almost everyone writing those articles is wrong

… or at the very least, missing the point.

Please don’t view this as me putting down the people who wrote those articles. I know a few of them personally, and I highly respect the work they are doing. This is not about whether their work is good or bad.

I’ve personally written many articles exactly like the ones they’re writing. In fact, they have one up on me because at least their articles are ranking in Google for popular search terms.

The reason they are missing the point is that most of those articles are focused on the elements of a page rather than the serving of a customer.

I get why they do it.

Webpages are far easier to understand than people. Webpages are a collection of 0s and 1s. People are a collection of who knows what.

And most of you, readers, are looking for webpage fixes — not a deeper, fuller way to serve your customer.

There is nothing necessarily wrong with you, but it’s just that we naturally focus on our own self-interest. It isn’t wrong in itself.

What is wrong is the methods we use to achieve our own goals. I don’t mean morally wrong. I mean practically wrong.

 

Our objective should always be: Make as much money possible.

MECLABS Institute has found after more than 15 years of research that the best method for achieving this objective is to spend as much money possible on serving your customer.

Until we can view every A/B test we run as an opportunity to better serve our customers, we will just be running (ultimately) ineffective tests on page elements.

It doesn’t really matter in the long run which color, layout or page element is going to perform well.

The Internet is constantly changing. Design trends are always going to influence how we look at webpages and their elements. What matters for marketers in the long run is how well we understand and, consequently, how well we can serve our customers.

Flint McGlaughlin, Managing Director and CEO, MECLABS, calls this understanding of our customers “customer wisdom.

This is also why he often says, “The goal of a test is not to get a lift, but rather to get a learning.”

However, it’s one thing to hear this, another to really understand what it means.

It really means we want to conduct research, not run a test.

We want to learn a tangible lesson about our customer so that we can apply it to other areas of our marketing and achieve a maximum return on the amount of time and energy we spend on testing.

Let me show you what I mean with a real-world example. Here’s what happens when you just run an A/B test that is focused on a page element. Let’s take color for instance.

You have two treatments. The only thing changed is the background color. 

 

You also have a result. In this case, the result was a 19.5% increase in clickthrough at a 92% level of confidence. But here’s where things get tricky.

Read more…

Permission Pass Email Send: A proven method for cleaning your mailing list

April 2nd, 2015 No comments

If you are reading this, you are likely in one of two positions:

  1. You have decided it is time to cleanse your email list of the inactive subscribers that no longer engage with your email sends, or …
  2. You need to stay compliant with your email management software (EMS), and you are being required to send your subscribers a permission pass to keep emailing them. A permission pass is a one-time send to an email list to reconfirm permission to email.

If you are in the latter position, don’t panic. This is actually a good opportunity to clean up your list and increase engagement with your current list.

At MarketingExperiments, our team recently did just that. We sent out a permission pass email to clean our list of inactive subscribers (which only drag down our rates).

We decided to run a test on the permission pass email based off of a previous blog that Daniel Burstein, Director of Editorial Content, MarketingSherpa, wrote back in September for a re-engagement campaign MarketingExperiments implemented after the Canadian Anti-Spam Legislation. While this campaign was not a permission pass, it was similar, and we were able to work off the findings from that campaign to formulate the test discussed in this blog post.

The main objective of the test was to see if subscribers would be more willing to opt back in with us if we offered them an incentive. While discovering that incentives were not valuable to inactive subscribers, our team also uncovered some valuable takeaways that will be quite insightful for any future permission pass sends.

 

Treatment #1. General Value

Treatment #1 focused on reminding subscribers of the value they would continue to receive with MarketingExperiments. 

 

Treatment #2. General Value and Incentive Offering

Treatment #2 also communicated a reminder of the value subscribers would continue to receive with MarketingExperiments. Additionally, it alerted them that by opting back in with MarketingExperiments, they would be entered to win a free MECLABS online training course.

Read more…