MarketingExperiments Blog: Research-driven optimization, testing, and marketing ideas

Homepages are vital to any website. Many times, they can be the first interaction our prospects have on our website and therefore what we do (or don’t do) at that moment can significantly impact our future relationship with them. But this is not anything that most marketer intuition hasn’t picked up already – hence the constant battle lines drawn over use of the homepage.

Where the marketing intuition gets a bit fuzzy is when it comes to strategically actualizing all the goals we have for a homepage with an optimal design. We know what we want a homepage to do, but we just don’t know exactly how to make it happen.

And this is where a recent experiment might give us a little insight on how well we understand home page design. Below are two homepages that were recently tested.

Could you pick the better page on pure gut alone?

Experiment Background:

This is a homepage test for a B2B company offering email marketing solutions for small and large businesses. The primary objective that we were attempting to increase conversion for was the free trial. Therefore, the key performance indicator (KPI) for this test will be total free trial starts.

As follows, the primary research question for this experiment is, “Which homepage will generate the most free trial sign-ups?” and the test we ran was an A/B multi-factorial split test utilizing our radical redesign methodologies.

Experiment designs (two versions):

Experiment Results (UPDATED):

Which was the winning homepage design? Well, we have revealed the results since the question was first posed in this post, and to many people’s surprise VERSION A produced 59% more free trials sign-ups than VERSION B. This difference is mostly accredited to a reduction in competing objectives, a simplified eye path, and an intensified communication of the value. What else do you think contributed to VERSION A’s success?

During Homepages Optimized: How using the homepage as a channel led to a 59% increase in conversion, we looked deeper into the results of the experiment above as well as discussed five key optimization strategies that should be utilized for every homepage.

Related Resources:

Homepages Optimized: How using the homepage as a channel led to a 59% increase in conversion

Homepage Optimization: How your peers use keywords and communicate with visitors

Research Report: Beyond Landing Pages

Clinic Notes

So often, beautiful design gets trumped by marketing objectives – and rightly so from a marketing perspective. The graphical elegance of a Web page might be worthy of an art exhibit, but if it doesn’t sell anything but “oohs” and “ahs,” what service does it really provide to anyone?

True, but it is at this point that we tend to divide. It’s them or us. Are you a marketer or a designer? Whose side are you on? As one popular design blog analyzes its relationship with marketing, “You can spell ‘team’ from the word ‘marketing,’ but I’ve yet to see a sense of it in marketing.”

But I think we (marketers) can and should live in both worlds. I believe design can be done in such a way as to actually contribute to the perceived value of an offer without being a distraction. I think marketing, whether they can measure it or not, is leaving money on the table when design is viewed as optional icing on the cake. Yes, I have a dream…

But, feelings aside, we must always default to testing – not our gut instincts. And so I was glad to see a recent experiment bring a little shimmer of hope to those of us who long for the day when these two often opposing worlds come together.

The Original Page (click to enlarge)

The original page was for an Australian company that offers dedicated hosting solutions. The primary goal of this page was to get the visitor to request a quote. These quotes were essentially leads that were then followed-up with and nurtured.

Nathan Thompson (who happens to be making his debut appearance on today’s Web clinic) was the key researcher for this page. Upon analysis, he identified the following three conversion threats:

1)  There is significant friction due to the volume of questions (many unnecessary) posed in the form.

2) There is not enough value being communicated with headlines, images, copy and CTA.

3) The overall low aesthetic quality of the design may be creating anxiety as to the actual capabilities of this company.

Of these issues, number three stood out as the most interesting hypothesis and I was eager to see the treatments he would put together to address this issue of design-induced anxiety.

The Optimized Page (click to enlarge)

The page that Nathan and the team created:

1) Significantly reduced the form fields required on the first step from 20 to four.

2)  Removed distracting banner images and strengthened the visibility of the headlines to better communicate the value.

3) Included a six-point expandable/interactive display of value proposition copy.

4) Added testimonials and cleaner, more modern images.

5) And then, finally, Nathan and the team focused on creating a more aesthetic design in order to improve the perceived credibility of this company. The thought was, if you are a business looking for a company with professional online capabilities, then a more professional design would generate better response. Let’s now go to the numbers.

The Results

Well, as the title reveals, the results of this newly designed page were significant. The new page outperformed the control by 188.46% with a statistical confidence level of 95%.

There are so many things that can be said about this test, and we will be drilling down much deeper on this experiment and others on today’s web clinic, but what is most notable to me right now is that the more graphical appealing page was the clear winner. This is different than what we have seen in past experimentation.

Now, we cannot forget that this is a radical redesign, multi-factorial test. There are many factors here being tested at once, and no doubt the results reflect that. But, with that said, all of these changes did happen within the context of a purposefully and strategically more aesthetic Web design. That’s noteworthy in my book. And for those who often feel the tension between marketing and design, it’s my peace offering for today.

What do you think?

Is this page more aesthetic or am I being too kind? How would you improve it? Did the more aesthetic design actually contribute to conversion or was it the other elements being tested?

Related Resources:

TODAY’S WEB CLINIC: Optimizing Landing Pages: The four key tactics that drove a 189% lift

The Creative vs. The Marketing Team: Yin And Yang, Oil And Water

Online Marketing and Advertising: How your peers provide and communicate value to customers

What “ugly duckling” landing pages can teach us

General

It has been a while since we have published Pay Per Click (PPC) ad testing. Lately, it can seem like PPC has taken the back seat to topics like social media and mobile. However, PPC remains an integral part of the online marketing campaigns for many of our own Research Partners, so we are still testing to discover the most effective ways to craft PPC ads.

So enough introductions and segues, let’s get down to the test.

Background

This experiment involves a business-to-business (B2B) software provider. They were trying to optimize one of their best lead generation conversion paths. They started by focusing their test efforts on the channel driving traffic through this process, the PPC ad.

At the end of the day, they would end up testing and optimizing the complete conversion path (from PPC ad to form page). And the overall gains this company achieved are going to turn some heads on today’s web clinic, but for the sake of this blog post, let’s just look at how they optimized their paid search marketing campaign.

The original PPC ad

The original ad used mainly vague qualitative statements (e.g., “award-winning,” “business software,” “fully integrated”) throughout, giving the ad little distinction from others.

Of course, there are limited characters within a PPC ad and it can be difficult to include deep specifics for certain claims and offers. But, as we will see from this test, sometimes it pays to get creative with our ad copy.

The new PPC ad

For the test, we were able to fit a bit more specific language on the second line of this ad. We included exact figures that underscored the popularity of this software. They have more than 6,459 current customers and their software is the top used by businesses.

It is important to note that, though “award-winning” could be expounded and specified, we had to balance the information we wanted to include in the ad with the amount of characters available. So we made sure that “award-winning” was explained immediately on the landing page customers reach by clicking on this ad.

The results

Overall we saw the click-through for the new ad increase 20.9% with no significant increase to the cost-per-click. For such a crowded market, an increase like this is nothing to turn your nose up at. This PPC was just one part of a holistic optimization strategy that, in the end, decreased the cost-per-acquisition by 66.4% and increased revenue by 267.9%.

So what’s does this mean for you?

This test underscores two key principles that we should all walk away with:

• First is the value of SPECIFICITY. Using clear statements provides a greater value perception in the mind of the user. If we want our PPC ads to stand out, we ought to use quantitative statements instead of (the much more common) qualitative claims. It’s a simple strategy, but it can have a significant impact on our efforts.
• Second is the importance of STRATEGY. This takes the complete picture to see clearly, but small gains are more powerful as part of a holistic optimization strategy. This 21% increase would be multiplied tenfold at the end of the day after we had optimized each step in the conversion funnel of this offer. So, when possible, test holistically.

Dr Flint McGlaughlin will be talking more about this second principle on today’s web clinic, as he walks through the entirety of this case study from PPC ad to the form submission page. He will explain in more detail how this company’s testing strategy took a 21% gain and multiplied it tenfold.

Related Resources

Compounding ROI of Sequential Conversion Rate Increases: How one company took a small gain and multiplied it tenfold

PPC Innovation: How will Google’s new lead capture extension affect your pay-per-click campaigns?

Face Your Fears: Why visitors really bounce from your site, part 2

Paid Search Marketing (PPC), Research Topics

Over the past couple of weeks, our content and production team has been mulling through research from multiple companies, trying to find the best experiments for today’s web clinic. It’s always exciting trying to find the right illustrative research that will serve our audience. In the midst of this, a very interesting case study came across my desk that I just had to share with our blog community.

The experiment came from the research library of ThinData 1:1 and was with a large airline that had a strong online presence. They had the goal of improving the level of user engagement (through higher click-through rates and lower unsubscribes) for their customer newsletters. They wanted to test a 1:1 marketing approach and see how much impact a personalized conversation with customers would have.

The Original Newsletter (click-to-enlarge)

First, it will be helpful to see what the original newsletter looked like. There are a few key things to note:

1) The goal of the email was to create enough interest on the part of the recipient to get them to click through to a specific flight offer.

2)  Every element on this email was sent generically to large subscriber segments and included offers without any reference to a recipient’s personal preferences.

3) The only (and it’s debatable) 1:1 marketing technique that the original email design included was the name of the recipient.

The 1:1 Marketing Strategy

One-to-one marketing conversation requires a couple of key ingredients, one of which is insight about your customers. So, to understand their customers better, this company created a “customer preference tool.” This tool took previous behavioral/purchase data and mapped out preferences in five key areas (destinations, airports, hotels, etc.). It also gave customers the ability to update their preferences for each of these areas.

So once they had some solid insight about the preferences of their newsletter recipients, they were ready to start a better conversation.

The New Design (click-to-enlarge)

A new email template was created to customize content according to the five key preference categories of the customers. In fact, the tool was inserted at the top of the email and functioned as a table of contents for the email. It also gave recipients a portal to their own preference center to make any changes.

The rest of the email was organized by an algorithm that gave higher placement to offers with higher preference status to the email recipients.

Instead of displaying all possible offers for a category, offers were filtered according to the preferences of the recipient.

Overall, the new email design created a 1:1 marketing conversation that matched the specific motivations of the recipients.

The Results

As mentioned in the title of the blog post, the results were impressive. For a large company that no doubt had already spent time improving these newsletters, they were able to increase the click-through rate generated from the emails by 21%. And this came by subtracting, not adding more links within the email. The new design also decreased unsubscribes by 60%, which the marketers interpreted as a big thumbs-up from their subscriber base.

Overall, the results of this test underscore the value and potential impact of striving to have a 1:1 conversation in our marketing campaigns.

What does this mean to you?

Ideally, you would have as much information as this company had about their customers, and if it were a perfect world, you would have unlimited technical development capabilities at your fingertips. Well, most of us don’t live in that perfect scenario and creating a true 1:1 marketing campaign like this company might seem a bit out of grasp.

Here’s the good news, 1:1 marketing has a path leading to it and everyone is somewhere on that path. And every marketer, regardless of company size, can improve the conversation they are having with their customers. Today on our web clinic, we will look at the strategic things that marketers of any level can do to improve their 1:1 marketing campaigns. So join us today as we talk about this case study and more during our free web clinic: 1-to-1 Marketing at Four Levels: Strategic ways every marketer can enter into an online conversation with customers

Note: To give credit where it is due, I’d like to point out again that the case study mentioned in this post came from the research library of of Thindata 1:1. Thindata 1:1 is a training partner of ours that provides one-to-one marketing automation solutions. They have done some interesting research in the area of one-to-one marketing and I would encourage you to check them out at www.thindata.com.

Clinic Notes, Email Marketing, Marketing Insights

Today on our web clinic – Technology Blind Spots: How human insight revealed a hidden (and almost missed) 31% gain – we will be releasing never before published research from our laboratory. And you know what we like to do with our audience when we have fresh research that they have never seen before…

That’s right; we like to turn them into guinea pigs.

We like to see if our blog readers, knowing the basic circumstance surrounding a recent test, can predict the outcome. How good is their online marketing radar? Can they spot a good webpage when they see one? How is marketing intuition performing these days?

But honestly, what really matters is the cheese they will be racing for today – one good-ole slice of free online certification course cheese with a little Twitter-love wine to wash it down.

Leave a comment below to enter and let the games begin.

The Experiment

The Research Partner we were working with provides online consumer brokerage services through a subscription-based model. This page, in particular, was aimed at visitors interested in signing up for the foreign exchange trading (FOREX) solutions.

The Control (click to zoom)

After analyzing the current landing page, we concluded that there were some significant factors contributing to confusion on this page. For one, there were  many competing graphical elements and objectives. In almost all cases, this type of layout negatively impacts conversion. We also believed that the value of this offer could be communicated with a little bit more oomph.

So we tested three designs against the control to address some of these issues.

Treatment 1 (click to zoom)

The first treatment is probably the closest to the control. However, there are some strategic changes.

First, we added a headline that better communicated the value of the offer. The copy also has been reorganized in a clearer, easier to read fashion.

And finally, we added a call-to-action button in the main section of copy.

Treatment 2 (click to zoom)

The next treatment used a more long copy approach than the control. It also incorporated a stronger headline and clearer copy layout similar to that of the first treatment.

It is important to note that some of the visual elements from the control have been removed from the bottom of the page. However, the left-hand column remained the same as the previous two designs.

Treatment 3 (click to zoom)

This version of the page is almost identical to treatment 2’s long copy layout.

The one big change for this version was that the elements in the left-hand column were changed into a simple navigation.

(Update) The Results

If you are reading this post now, the contest mentioned above is over. Congrats to @terryrydzynski, a marketer who’s intuition got him a free seat in one of our online certification courses. If I were you, I’d follow this brilliant guy’s twitter account.

Which one was the winner you ask? All of the treatments outperformed the control, but Treatment 3 had the highest conversion rate with a validated 31% increase over the control. Now the results were not too surprising if you read some of the reasoning behind our designs above. Treatment 3 significantly reduced the amount of friction over the control by removing the competing graphical elements and focusing the visitor on one objective.

So what can we learn from this experiment?

If there’s one thing that we can all take away from this case study, it is that many times we are trying to accomplish way too much with our pages, and if we could just simplify our message and make options clear for our visitors, we would potentially see an increase in response.

But this is just scratching the surface, if you would like more information about this case study and some of it’s implications, you can find a more detailed explanation in the replay of yesterday’s web clinic, which will be available next week. To be notified when the replay is available, feel free to sign up for free research updates from MarketingExperiments.

Analytics & Testing, Clinic Notes, Landing Page Optimization

One of my favorite aspects of working in a family of research companies is getting to see new theories tested. It is a core essence of our “Discover what works” tagline, and when a recent “discovery” experiment came across my desk the other day, I couldn’t help but want to share its implications with our audience. Let me explain the background and give you a little preview of the discovery that will be fully revealed in today’s web clinic.

What’s the story?

We are currently running tests with a large subscription-based news service that has a large direct mail (DM) aspect to their marketing campaigns. We saw this as an opportunity to test some of the hypotheses we have been formulating related to the connection between offline and online efforts.

Now, as with most DM efforts, their campaigns had a call-to-action that included an online option. For those interested in signing up for the offer online, they could go to a specific URL. The URL was unique, the path was clear, and the impact of the direct mail campaign would be easy to track. Right? Well…

Those DM rebels

We wondered how many DM recipients were wandering off the beaten path of a nice clean and trackable vanity URL. We questioned how many people who received the offline campaign were rebelling against the directions and forging their own way to what they wanted. And overall, how much of this company’s generic website traffic was taking part in this underground rebellion?

All these questions led us to test the notion that DM pieces might be having a greater impact than planned for and that we might be underestimating the significant opportunity created by our offline efforts

How we set up the test

So we set up a split test that would integrate some of the messaging from the offline DM campaigns into a generic, but similar product offer page on their website. The elements we tested initially were mainly images. The DM pieces relied on strong images and, to create continuity, we used similar images on the generic website pages (see below). This test was also specifically set to run during the time that the direct mail piece would be landing in mail boxes. (out of courtesy to the Research Partner, we have anonymized these pieces):

Original Offer Page:               Direct Mail Offer:                New Offer Page:

Our thought here was that if there was any traffic coming from the direct mail to these generic offer pages, that the images would increase relevance and have a positive impact on conversion if many people were coming from the DM campaigns. We were hoping to see a difference in the results over time related to the offline campaign.

The Results – A 124% increase in subscriptions

The results were stunning. During the two weeks that the DM was out, we saw a 124% lift in subscriptions for the new page. After the two weeks, however, the conversion rates for the page that integrated imagery from the DM campaign went back down to its historical average.

This supported our hypothesis that offline efforts have a larger impact than might be expected and that people don’t necessarily follow the prescribed path of a DM campaign. For this company, many ended up on generic product pages.

What does this test mean for all of us? I believe we can learn two things:  1) If you are running offline campaigns, be aware that they may be having a larger impact than you think and 2) Identifying where response to your offline campaigns overlaps with your online visitor’s engagement, and establishing a connection between the two messages can generate significant response.

For this company, discovering that overlap and connecting the messages meant 124% more subscriptions. What might it mean for the rest of us?

For a deeper explanation and analysis of this experiment, activate your FREE MarketingExperiments email subscription to be notified when the replay of this web clinic is available.

Related Resources

Integrate Your Marketing: How one company combined offline and online marketing to increase subscriptions

Do you support your web site through offline media?

Should offline and online design elements be the same?

Clinic Notes