Ken Bowen

Live From MarketingSherpa Summit 2016: Humana on the power of iterative testing

February 22nd, 2016

This is Ken Bowen, reporting live from the Bellagio hotel in beautiful Las Vegas for MarketingSherpa Summit 2016. For over a decade, Email Summit has been one of the most exciting weeks of the year for email marketing practitioners.

This year, just when you think you’ve got it all figured out, we’ve gone and changed the rules.

Instead of just email, we’ve expanded our focus to also include content marketing, social media, mobile and data.

As Managing Editor of MarketingExperiments, MarketingSherpa’s sister site, I’ve had the privilege of working closely with all nine presenters on the Digital & Data track over the last several months. These marketing leaders are on the bleeding edge of disciplines like behavioral economics, account based-marketing, audience and demand generation, individualized marketing, customer satisfaction quantification and even data-driven direct mail.

It’s been a wild couple of months leading up to the event — full of late nights at the office, jamming brainstorming sessions into every break and those crazy evenings where you shoot out of bed at 3:00 a.m. with the perfect slide idea. The exclamation point came yesterday, when a modest flight from our home base in Jacksonville, Florida to Las Vegas managed to turn into a 22-hour Homeric epic that played out like a mix of The Terminal and Home Alone.

And I’ve loved every minute of it.

After spending months getting to learn from and know some of the brightest, most forward-thinking minds in digital marketing, when I popped out of bed this morning and saw this view out the window, it felt like Christmas morning in the desert, knowing that we would soon get the chance to share these great case studies and stories with our readers.


We hope to bring you details and video of each presentation in the coming weeks, but for now, let’s take a look at how Mike Loveridge, Head of Digital Test and Learn, Humana, and his team leveraged iterative testing and captured big gains.


Iterative Testing

“We’re trying to change customer perception from ‘we’re a big bad insurance company that’s trying to take your money and deny your claims’ to ‘we want to be your health partner for life,’” Mike Loveridge said to the overflow crowd in the Da Vinci ballroom.

“Our goal is to improve customer’s health by 20% by 2020,” he added.

One way to shift that perception is by testing your messaging.

“[Testing radical redesign] is impractical,” Mike said. “Testing complete new versus complete old was a nightmare.”

Instead, Humana starting testing iteratively, with an end goal of testing the former old funnel versus the complete new funnel, optimized step-by-step, to see if sum of the parts equals the whole. Mike referred to this as Humana’s “Super Test” strategy.


During the presentation, Mike walked us through several of these iterative tests. We’re excited to share the full video with you soon, but in the meantime, let’s take a look at three of the upper-funnel banner tests that Mike presented.


Test #1: Banner redesign

For this banner test — which Mike jokingly referred to as “Winter grandma” versus “Summer Clint Eastwood lookalike” — the team at Humana wanted to test whether the messaging being used on the rotating banner of its homepage was too complex for customers.


“You really have to simplify when you only have six seconds,” Mike said.

In addition to the change in imagery, the headline was better tailored to users’ position in the customer journey (upper funnel), the copy was simplified and the CTA was given more visual punch.

The result?

A 433% increase in clickthrough into Humana’s marketing funnel. By serving customers with what they want, when they need it, in an accessible way, banner performance skyrocketed.


Test #2: GEO personalization

In this test, the Humana team tested geo personalization, which serves unique pages, ads, etc. based on user location.


“Take a couple, put them in a background that’s fairly generic, it could be anywhere in the world,” Mike said in reference to the Control tested in this experiment. “Then take that couple, and put them in front of the Statue of Liberty or the Golden Gate Bridge.”

More specifically, the banner featuring the Golden Gate bridge, as pictured above, was served to site users located in California’s Bay Area. And, as you’d expect, the banner featuring the Statue of Liberty was delivered to New York.

The result?

A 31% increase in clickthrough.

But this wasn’t the biggest surprise.

Mike and the Humana team found that geo-personalization not only increased clickthrough, but also increased overall conversion through the funnel.


Test #3: Banner navigation

“For this next banner test,” Mike told the audience, “we simplified the title, bumped up the font size and made the supporting text even simpler. There was a lot of green on the page, so we made the button plum; it really stood out.”

The Humana team also changed the scrolling navigation counter (bottom right corner, above) from plum to grey, so that it wasn’t competing for attention with the CTA.

The result?

The Treatment outperformed the Control by 192% in clickthrough to the next step in the funnel.

Providing further clarity to the customer, both visually and via text, resulted in a massive boost in banner performance.


Key Takeaway

Later in the presentation, Mike also shared some deeper funnel tests, and spoke in-depth about the necessity of communicating the value of testing both top-down and bottom-up in order to gain buy-in within an organization.

But if there’s one thing that Mike wants marketers to take away, it’s the sheer power of iterative testing.

“Iterative testing gives you more results on a more frequent basis,” he said. “And they’re easier to understand.”

Mike is just one of the 30+ speakers who will have presented by the time the sun sets on MarketingSherpa Summit 2016 tomorrow evening.

To read about (and see) more of these incredible stories from our fellow marketers, stay tuned here at MarketingExperiments, and also at our sister site MarketingSherpa.

In the meantime, it’s been an honor getting to meet so many of you here in Las Vegas and to hear about your own customer-first marketing success stories.


You might also like

Email Marketing: Preheader testing generates 30% higher newsletter open rate for trade journal

3 Takeaways from the MECLABS Live Optimization Webinar

Value Prop: Three Super Bowl commercials that nailed the “Only-Factor”

Ken Bowen

About Ken Bowen

Ken Bowen, Managing Editor, MarketingExperiments As Managing Editor of MarketingExperiments, Ken’s goal is to transform raw test data into clear, accurate, actionable content for our readers. Prior to joining MECLABS, Ken worked in civil engineering, interfacing with clients such as NASA, the Florida Department of Transportation, the Jacksonville Transportation Authority and UF Shands Medical Center. Ken has a degree in Economics and recently released a book, Big League City! 100 Years of Football in Jacksonville, with a foreword by Jacksonville Jaguars’ owner Shad Khan. In his limited spare time, Ken enjoys Jacksonville Jaguars’ football, Chicago Cubs baseball, TV, and general pop-culture. Though Ken is a proud Seminole, his true loyalty lies with the Clemson Tigers. In particular, Ken idolizes Dabo Swinney, the greatest coach in college football history.

Categories: Events Tags:

We no longer accept comments on the MarketingExperiments blog, but we'd love to hear what you've learned about customer-first marketing. Send us a Letter to the Editor to share your story.