Your customers may be flat out ignoring your latest news, offers, and ads. Don’t blame them. It’s simple human nature.

Take a quick look at your surroundings – your cubicle, your office, your solarium – wherever you’re reading this. How much do you notice what’s around you? I mean…really notice?

Not as much as you think you do, I’m guessing. Take a recent experiment run here in the labs. And by “experiment” I mean “practical joke run by our Associate Director of Optimization, Adam Lapp.”

Adam Photoshopped a picture of one of our Research Analysts, Ashley, posing with a friend. It’s the picture in this blog post. Perhaps it looks normal at first glance, but if you take a closer look, you can see that the blonde woman on the right looks a little, well, masculine.

That’s because Adam Photoshopped the face of a male Research Analyst over the face of Ashley’s female friend. He then replaced the photo she had hanging in her cubicle with this photo.

And, Ashley didn’t even notice her friend’s metamorphosis until someone pointed it out to her. Even though it was right in front of her face all day. Why?

-

People don’t notice subtle changes in familiar environments

Now I don’t want to throw Ashley under the bus. I fully admit, I’m no better (and neither are you…or our customers).

For example, we recently moved the official offices of MarketingSherpa from Rhode Island to right here in Jacksonville Beach, Florida. I’ve been very cognizant of the need to look for where changes of that address need to be made on our many Web properties.

But I didn’t notice that the change wasn’t made on the MarketingSherpa Twitter page – even though I look at it at least five times a day.

However, when I was interviewing Ryan Amirault, Digital Marketing Manager, Whole Foods Market, for his case study at Email Summit 2012, he instantly noticed the location and started talking about Rhode Island.

So while being new to, say, a landing page, makes the customer more likely to notice the discreet marketing message, even novelty may not help…

-

Invisible Gorilla Test

What we’re really talking about when we say “banner blindness” is a phenomenon scientists refer to as “inattentional blindness” or “selective attention.” The typical person is overloaded with visual stimuli and inputs of all sorts, making it simply impossible to focus on everything. So, people often overlook things that are right in front of their face.

One of the most famous examples of this is the Invisible Gorilla Test…

-

Now, I may have already primed you to see the gorilla in the above video. However, when Daniel Simons of the University of Illinois at Urbana-Champaign, and Christopher Chabris of Harvard University ran this test, they found that, in most test subject groups, 50% of the subjects did not report seeing the gorilla.

-

How can you get your customers to see your ads?

So if a gorilla doesn’t work, you might be thinking , “I need to go one step further. From now on, every ad I run will have a tap dancing chimpanzee with a neon tracksuit.”

Easy, big fella. You need not become a carnival barker to grab your audience’s attention. All I want to draw your attention to is the fact that what is obvious to you (since you likely eat, sleep, and breathe your marketing message) is not always readily apparent to your audience. And don’t take for granted that your message got across just because you put it at the top of your homepage.

Here are a few common sense thoughts to keep in mind as you seek to overcome banner blindness:

-

Contrast: The more an object can stick out, due to bright colors, crazy patterns, or motion, the more people are likely to notice it. (Keep in mind, if you are running a pay-per-click ad, you want more than attention and curiosity clicks … you want quality clicks.)

Here is an example from the MarketingSherpa site. There is a clear contrast between the ad and its surroundings, thanks to the different color, the bright visual, and even in how “Reserve Your Seat” interrupts the “Limited seating still available” rectangle (please note, it comes out a little brighter on the screen than it does on the screen capture below).

-

Click to enlarge

-

A Multichannel Approach: Don’t think that just placing a message at the top of your homepage, smack dab in the middle (especially in a similar font as its surroundings), is enough to get your message across. I’ve made this mistake myself before. “What do you mean we didn’t tell people? It’s right there at the top of the homepage.”Most people won’t even see it. Especially if they visit your site often.

That doesn’t mean you can’t put your message there, it just means to reach your potential customers in as many way as possible with the message – email, social media, dedicated landing pages, offline communications – and not take for granted that the message was received just by placing it on your homepage.

-

Get in the Middle: There are certain areas of webpages that most people usually relegate to background noise – the top and bottom headers, the right and left columns. When placing an ad, or putting information on your own site, try to get right into the middle of the content.

If you aren’t able to, try to make sure your information is at least at a natural stopping point for the content – for example, just to the right of or below the end of a blog post or article.

-

Break out of the Box: Many marketers place information in a box on their homepages or landing pages that doesn’t necessarily need to be in that box…and therefore their audience is overlooking it.

From a headline on a homepage that is placed in a box (and therefore ignored) to testimonials that look like text-based ads, this mistake is all too common. When you’re on your own website, make sure you are not inadvertently making important information look like a banner ad that will be — you guessed it — totally overlooked by your visitors.

-

Related Resources:

The Ultimate Click: How to get what you pay for with pay-per-click advertising

Banner Blindness: Optimize your online display advertising to stick out (or blend in)

Online Advertising: The 3 obstacles you must overcome to create an effective banner ad

Banner Ad Design: The 3 key banner objectives that drove a 285% lift

Banner Design Tested: How a 35% decrease in clicks caused an 88% increase in conversion

According to the MarketingExperiments’ conversion heuristic, anxiety is one negative factor that reduces the likelihood that your potential customers will complete that lead form or buy from you. One of the chief causes of anxiety for customers of late has been privacy.

For example, 94% of 45-64 year olds think there should be a law that requires websites and advertising companies to delete all stored information about an individual, according to research conducted by the University of California at Berkeley and the University of Pennsylvania.

And you likely see more headlines every day. The Wall Street Journal has even been conducting a yearlong investigative reporting series titled “What They Know.”

And where there is customer sentiment, there is opportunity

So what if, instead of only responding to regulations and industry edicts, you became proactive with your products and services? What if you did such a good job of reducing customer anxiety around privacy, you turned it into a competitive advantage for your company? Here are a few ideas to get you started (and I’d love to hear yours as well in the comment section) …

Transparency reduces anxiety

Address the elephant in the room and clearly communicate with your audience about how you use their information. This should go far beyond a privacy policy hidden somewhere in your footer.

For example, researchers at Carnegie Mellon University created a shopping search engine interface that clearly displayed privacy policy information. They dubbed it Privacy Finder …

Click to enlarge

Their research found that some consumers are willing to pay a premium to purchase from more privacy protective websites.

But you don’t need a fancy, whiz bang technology solution to reduce customer anxiety about privacy. Here’s an example email I received courtesy of Melissa Miller, Inbound Marketing Manager, HubSpot.

“Beyond acknowledging the fact that their audience is concerned about privacy, they explain that their users have complete control of what they share,” she said. “I thought it was a good message … that they are letting me decide what I do and don’t want to share.”

Here’s the email copy …

Subject Line: How Rypple Protects Your Privacy

 

Hi Melissa,
Social networks have developed a stigma: people worry that every word they type will be visible to everyone else.

Rypple is different. It’s been carefully designed for the workplace, so users can control how public—or private—they want the shared information to be.

We gave New York Times Gadgetwise blogger Paul Boutin full access to Rypple’s social performance management platform.  Today he shares what he’s learned about how Rypple helps you use the power of a social network to connect with your co-workers — while also ensuring that your private communications stay private.

You will learn how to:

• recognize a colleague for great work and make it public to the whole company
• share private notes visible only to your manager or direct report
• ask for anonymous feedback about your performance

How Rypple Protects Your Privacy

Sincerely,

Daniel Debow
co-CEO, Rypple

You can see from the email’s last link that Rypple even hired a freelance writer to pen a blog post. Why? Rypple understands that customer anxiety about privacy could hurt sales.

“While the need for privacy may not be a company’s primary reason for buying new enterprise software, it can become an insurmountable roadblock if that need isn’t satisfied,” said Nick Stein, Director, Content & Media, Rypple.

Burn the cookies (publicly)

While we might not have a telescreen in every house, cookies are the embodiment of Big Brother as envisioned by George Orwell. (Or was it Steve Jobs and Chiat Day?)

They watch … and track … and report on everything the average customer does. And they are omnipresent. According to The Wall Street Journal, America’s top 50 websites installed 64 pieces of tracking technology onto visitor’s computers, on average. A dozen sites each installed more than 100! The lone holdout? Care to guess? (Answer is below.)

So what if you didn’t install any cookies? And what if you made that part of your value proposition with a big “No Cookie Promise” seal on your site? What approach could you take to target customers with compelling offers? Well, tracking customers across the Internet is not the only way to deliver relevant offers …

“Next year, we’ll be undertaking a campaign for a large financial services company. Finance is a particular topic where privacy concerns come into play,” said Dave Hills, CEO, Twelvefold Media. “Without any tracking, we’ll be able to execute a ‘conditional marketing campaign.’ In other words, being able to deliver ‘bull’ or ‘bear’ messages depending on what the market is doing.”

“Aside from privacy, this is all about delivering content relevant to the audience at an exact point in time – not yesterday, not tomorrow.”

And the only site among the top 50 that didn’t install any cookies – Wikipedia.

Beyond the landing page

App store optimization is the latest, bleeding edge battle ground among marketers. How do you convince customers to pay for your app when there are so many similar competitors? Even if your customers don’t have to pay any money for the app, anxiety may still discourage a download.

According to a recent study by AdaptiveMobile, 75%of respondents said they would pay more for apps they thought guarded their privacy better.

In past landing page tests conducted by MarketingExperiments, we’ve found that third-party privacy seals help reduce anxiety and improve conversion. Not surprisingly, TRUSTe has found similar results and is looking to supply its seal to app developers as well.

“Consumers are more likely to engage online with businesses they trust,” said Dave Deasy, Vice President, Marketing, TRUSTe. “The benefit of having a third party vouch for the legitimacy of your privacy practices can have great returns.” For example, TRUSTe measured an 84% increase in purchases for wedding site Thepros.com.

Dave went on to say … “If a business has taken the steps to ensure their privacy practices are to the highest standard, the best way to use this as a marketing vehicle is to make this effort know to the consumer.”

Well said. So if you’re doing right by your customers, let them know. Whether it’s through a snazzy interface, a third-party seal, or a simple email, blog post, or clear copy on your site, reducing customer anxiety can increase sales … and give your company a competitive advantage.

Related Resources:

Landing Page Optimization: Addressing customer anxiety

B2B Marketing: 5 privacy factors to consider when using marketing automation

Online Behavioral Advertising: How to benefit from targeted ads in a world concerned with privacy

Twelvefold Media

AdaptiveMobile

After driving potential customers to your website through PPC ads, organic search results, or a targeted campaign to create online traffic, the most logical way to turn those visitors into leads is by getting them to register via an online form.

For this blog post, let’s assume you’ve already presented an offer with a strong enough value proposition to get your visitor to actually be interested in filling out your registration form. Let’s just focus on optimizing the form itself.

A form can have any number of fields, of course, and it would seem if you have more required fields on the form, you will generate more lead information for the database.

Can we all agree to that point? Great, let’s take a look at a recent lead generation test then.

Background

Marketo, the marketing automation company, has a content-fueled top-of-the-funnel marketing strategy that drives traffic to its website in order to access that content. Some of the content is ungated, but what the company calls “middle stage” content (buyer’s guides, ROI calculator, RFP template, independent analyst research, etc.) and some of its “late stage” content (such as a comprehensive product demo) all require registration.

Because Marketo requires what it considers high-quality website visitors to register so it can track their behavior and contact those visitors, it conducted A/B testing on its forms to find the best-performing version.

Treatments

The company tested forms with five, seven and nine field forms.

The five-field form asked for:

• First Name
• Last Name
• Work Email
• Job Function (with dropdown menu)
• Company

All five fields are required.

The seven-field form added “# Employees” and “Industry,” both with dropdown menus.

The nine-field form added “Work Phone” and “CRM System.”

Results

The short form with only five fields — first and last name, work email, job function and company — produced an average conversion of 13.4 percent, with a cost-per-lead of $31.24.

Adding additional company information fields — number of employees and industry — to the form dropped conversion to 12 percent and raised the cost-per-lead to $34.94.

Adding CRM system and work phone fields further lowered conversion to 10% and cost-per-lead jumped to $41.90.

Click to enlarge

Clearly, shorter forms are more effective in getting Web visitors to actually complete the registration and in terms of lowering the cost-per-lead. The trade-off is that with a shorter form is you are collecting less information for your database.

Jon Miller, VP Marketing, Marketo, doesn’t see using the more effective shorter forms as a problem.

“People lie on the forms all the time,” he says. MarketingSherpa research backs up Jon’s sentiment, showing the accuracy rate of self-submitted data.

Click to enlarge

Jon’s solution is to use the basic information collected on the short form and buy the data to fill out the database record on the new lead.

He explains, for Marketo it is actually less expensive to use a short registration form with a higher conversion and lower cost-per-lead to capture basic lead information that can then be used in conjunction with an outside vendor to complete the record, than it is to ask for more information on the registration form. Information that might not be correct, anyway.

Jon sums up the results of Marketo’s form field testing, “Why should I ask for data, hurt my conversion rate and get bad data when I can buy it cheaper?”

Related Resources

Jigsaw (Marketo’s database vendor)

Members library — Marketing Strategy: Revenue-oriented approach leads to 700% two-year growth

Internet Marketing: Optimizing form fields to maximize conversions

Marketing Strategies: Is performance-based vendor pricing the best value?

Lead Marketing: Cost-per-lead and lead nurturing ROI

Interview: 4 Tactics to Lift Product Registration Forms 20% or More

It was time to get radical at Senior Optimization Manager, Adam Lapp’s optimization class, and once again I was there to document it all for your reading pleasure. Because I care, naturally.

Since my last post, the student body has grown, and we now have five new research analysts eager to learn. Due to our uneven number, Adam decided to pit all of the analysts against each other for his optimization competition, instead of taking the usual team approach. This time around, their challenge was to create the most effective radical redesign for the Arbor Day Foundation homepage (which was submitted for live optimization during our Homepage Web clinic).

But before I show you the original page, I’d like to add that this lesson not only sharpened our optimization skills, but also proved that great minds don’t always think alike. And, believe it or not that can be a really good thing.

Diagnosing the homepage

Now, let’s get to down the “nitty gritty”…here’s the audience submission:

Click to enlarge

Before being given the task to create radical wireframes for the original homepage, Adam and the class discussed the page’s main issues. Here is their analysis the key problems visitors might face:

• The logo is too small. It gets lost in the page and also doesn’t help the visitor know exactly where they are
• There’s not a clear focus on the page

o   It has a confusing top navigation

o   It has three equally weighted columns (Trees, Programs, Lend Your Support)

o   It also has confusing objectives (Where do I click, what can I do on this page?)

1. What is the difference between a membership, a donation and buying a tree?

• Unclear call-to-action

o   The first and largest call-to-action a visitor sees is “Visit the tree nursery”

o   Makes the visitor ask, “What is a tree nursery? Is this a place to buy trees or a clever metaphor for a type of product or page?”

o   Visitors have unclear expectations of what the next page will be and are a little lost at this point.

Radical Solutions

After pointing out these issues, the analysts had to figure out how to tie in all these objectives together into one goal. And when it was time to present their radical wireframes, each analyst came up with a completely different design and goal in mind (remember the original page had several objectives).

Each submitted wireframe focused on one of the following goals:

Click to enlarge

-Directly pushing a membership

• The headline is action oriented, specific and includes a membership incentive
• It drives the visitor’s attention to a primary call-to-action (become a member)

Click to enlarge

-Getting visitors to donate and help the Arbor Day Foundation’s mission

• Headline immediately says where you are
• The options on left let visitors know what to do on the page (donate/help)

Click to enlarge

- Giving the gift of a tree or allowing visitors to pick 10 trees for themselves now

• Page objective is simplified and the incentive of 10 free gifts with donation is emphasized
• De-emphasized supporting column with core information about the foundation

Click to enlarge

-Engaging visitors with an interactive game

• Headline tells visitors exactly where they are
• Focuses on letting visitors interact and become familiar with the product through question game

Click to enlarge

-Pushing all objectives in a new navigation-focused design

• Visitor can quickly choose a path from the easy to see, simple, and short left navigation on the top portion.
• Has a rotating banner will draw the eye path to the four most important objectives, then presents four core values in a controllable (and testable) sequence

Five heads are better than one

These five separate takes on one Web page got us thinking about the brainstorming process and how important it is for marketers to create a blissful marriage between individual ideas and teamwork.

In a marketing team, not everyone is going to think the same way, but this doesn’t mean that these differing thoughts can’t turn into a great collaborative effort.

This reminds me of the weekly peer review meetings we have at out lab, where the research team gets together and reviews Web pages as a team. Usually, one person starts off the conversation and then another person bounces off an idea about that topic, sometimes agreeing or challenging the previous comment. The idea here is to brainstorm in the correct way, by being open to different ideas and avoiding groupthink. Of course, not every idea is a good idea, which is why you have to test.

This is exactly what happened during Adam’s class. Each analyst initially came in with a different idea for the homepage and after everyone’s thoughts were put on the table, each individual thought grew into one improved and cohesive approach to developing treatments.

“If one person in charge of this page only relied on their own ideas, we may not [be] able to achieve the highest level of success,” Adam said. Of course, this hypothetical collaborative effort would have to be tested to make sure it’s actually effective.

“There’s no such thing as expert marketers, only expert testers,” Adam said. “AND the best way to get diverse test ideas is to leverage other people with other ideas.”

Lesson learned

In the end, even though it started as a free-for-all competition, with the right guidance and frame of mind, it turned out to be a great team effort.

Now that we’ve learned that lesson, I would love to start a collaborative effort with the audience, and ask you to share your ideas to optimize this homepage. Feel free to use the comments section to get this brainstorming session rolling.

Related Resources

Homepage Optimization: Radical redesign ideas for multivariable testing

Homepage Optimization: Creating the best design to quickly meet multiple visitors’ needs

Informed Dissent: The best marketing campaigns come from the best ideas

Landing Page Optimization: What cyclical products can learn from CBS Sports

How often have you placed an item in your shopping cart in a bricks and mortar store, a can of green beans for example, and then changed your mind before you forked over the cash and put those green beans back on the shelf without buying?

How about online? Ever drop something in the ol’ virtual shopping cart, but change your mind and decide not to buy?

If you’re like me, it’s very rare for you to change your mind in a physical store, but I do it online all the time. Why? And how can you reduce shopping cart abandonment and improve product pages on your own site? There is no one right answer to these very difficult questions, but we’re going to be sharing our latest research discoveries today at 4 p.m. EDT in our latest free Web clinic – Shopping Carts Optimized: How a few tweaks led to 12% more revenue across an entire e-commerce website.

But first, we asked your peers these vexing questions. Below are a few of our favorite responses…

Five factors to optimize

Shopping cart conversion rates and success of the order checkout process depends on many business-specific factors, such as type of products, overall e-commerce site quality, maturity of the business, and transparency and visibility of shipping, return and security policies.

There are requirements which are de facto standard for any e-commerce web site, such as overall performance (order checkout forms must load under 10 seconds and a shopping cart that should work when cookies are disabled.

But there are also some tricks which can improve shopping cart conversion rates, such as:

1. Keep promoting items in the basket list

Always display an image thumbnail of the product/service in the basket list. That alone can cut the cart abandonment rate in half. Display the time and date when an item is added. In the case of price reduction or promotion/discount – always display how much money will be saved.

2. Security, Payment, Shipping.

Answers to four main questions are critical:

• How secure and private is the order checkout?
• What payment methods are accepted?
• Where can the items can be shipped?
• What is the cost of the shipping?

These answers must be clearly and visually presented on every page of the e-commerce site (in the footer) and in the basket page next to the order checkout button (icons of the SSL certificate provider, accepted payment methods, and shipping carriers with pop-up shipping rate cards usually does the trick).

3. Keep your basket and order checkout simple.

Avoid AJAX, Flash, etc. You have already impressed a visitor to the extent that he or she wants to buy something from you. Don’t risk losing the sale, because your visitor might be using an older browser or specific device, which doesn’t support your “cool” features.

4. Less distraction – prompt to action.

Show cross- or up-sell products in the basket with enough information to make a purchase decision and add them to the basket without the need to leave the page. Form input for voucher/discount/gift card codes should be located on top of the “total” line of the basket.

5. Shadow account registration

Make the visitor feel that the registration is a value-added benefit, even though it is mandatory. Collect all necessary data from the order checkout and ask to create a password in order to track the order progress at the order confirmation stage.

– Andrei Vesselovski, CEO, Netcomposite

Optimize and test the product page

The greatest difficulty with product page design is that these pages have to cater to different informational needs. Primarily, those of a person who has already decided to buy and wants to be fully supported through a simple transparent shopping cart transaction and those of a person who is undecided and needs to have their feelings satisfied and questions answered.

In the absence of being able to touch products, customers need to have greater access to substitute sensory experiences. This includes having an extensive photo library that allows the item to be viewed from every angle as well as macro views [for very close inspection of manufacture and details]. Consideration also needs to be given to whether adding audio or video recordings that demonstrate actual use of the product (for example, giving proof on things such as silent running[noise generated], ease of use or installation etc.) is of value for decision makers.

Whilst these additional ways of interacting aren’t necessarily essential for making the sale, and in many cases will remain unused, they add significantly to the overall value of credibility and trust attributed to the website and business (also great for market differentiation) based on visitor feedback.

In terms of the design of your product page, the trick is to balance the two information needs and keep the focus on what your outcome is supposed to be – that means having a clean simple design and providing access into these additional experiential resources on demand (through clicks) so that you aren’t detracting from the buying message.

How much you display in the product page view and what can be made an additional click away is absolutely dependent on your product/market combination and must be tested.

Testing is also required to ensure that additional links on your page don’t detract from your call to action (add to cart). So the manner in which they are designed and displayed (as links, icons, buttons or clickable images) must be tested (I’m often surprised how the outcomes of testing are diametrically opposed to what I’d expect.

When it comes to the shopping cart part of the buying experience, my tip is to ensure you maintain state across multiple browser window sessions, so that when they open previews in new windows (or tabs) whatever they do impacts on the cart. That means when they close that preview window and go to another page/window/tab, the cart has been update/refreshed with their action from the just closed window. Basically, maintaining consistency in expectation from the visitor’s perspective.

– Indra Chandon, Manager, Solutions Answers Results

Don’t overlook great content

Two items are extremely important:

1. Customer reviews and ratings. Consumers look at other people’s experiences with a product and this can have a great impact on your sales. Amazon proves this point.

2. Great product descriptions. People come to the Web looking for information, so give it to them! Provide enough detail that they can make a buying decision. Provide complete specifications, videos, etc. Look at TigerDirect.com as an example. But remember, you are the salesperson, so don’t just repeat the manufacturer’s canned description – tell the customer why you personally recommend this product and let them know how it will help them.

Basically, these two things boil down to content! Provide great content and everything else is just icing on the cake.

– Greg Jameson, President, WebStores Ltd

Related Resources

E-commerce Shopping Carts: How a redesigned checkout process led to 13% increase in conversion rate

Shopping Cart Abandonment: How not being annoying can get you 67% more cart completions

What Else Can I Test to Reduce Shopping Cart Abandonment Rate?

E-commerce: Using multivariate testing to increase sales 83.79%

At the end of March, HubSpot, the marketing software company, held a webinar on “The Science of Timing.” The event drew almost 25,000 registrants and ended up with very close to 10,000 attendees. Driving this success was a fast-paced and extensive email campaign sent to different segments of HubSpot’s list, prospects and new leads.

“The Science of Timing” was hosted by Dan Zarrella, HubSpot’s Social Media Scientist, and was built on a previous series of webinars over the last two years covering topics such as “The Science of Twitter,” “The Science of Blogging,” “The Science of Facebook,” and more. “The Science of Timing” covered when to do these social activities — when to tweet when seeking a retweet, when to publish new blog posts, when to update a Facebook status.

Subject line testing

HubSpot regularly conducts testing and optimization, and for this send it tested the subject line.

Eric Vreeland, Inbound Marketing Manager, HubSpot, explains the reason for this test, “Our goal is to generate the most amount of leads possible. Because our email list is so large, a 0.1% increase in open or clickthrough can result in a large number of form conversions and leads. We opted to test subject lines in order to maximize the total conversions we got for all our promotional efforts.”

To create a mailing list for the test, HubSpot randomly pulled about 50,000 addresses from its total list of around 500,000. This subgroup was randomly broken into five groups, and each group was sent mail with the exact same copy:

Click to enlarge

And all five groups were sent to the landing page after clicking through:

Click to enlarge

All five sends went out simultaneously at 10:00 a.m. The total list size was 54,987.

Results

Group 1

• Subject line: [New Webinar] What time should you blog, tweet, and email?
• Total delivered: 11,037
• Total opens: 1,534
• Total clicks: 245
• Open rate: 13.90%
• Clickthrough: 2.22%

Group 2

• Subject line: [New Data] The Science of Timing. When you should tweet, blog, email, and more
• Total delivered: 10,996
• Total opens: 1,562
• Total clicks: 247
• Open rate: 14.21%
• Clickthrough: 2.25%

Group 3

• Subject line: [New Webinar] The best times to do your online marketing
• Total delivered: 10,924
• Total opens: 1,506
• Total clicks: 214
• Open rate: 13.79%
• Clickthrough: 1.96%

Group 4

• Subject line: [New Webinar] Learn the science of timing your blog posts, tweets, emails and more
• Total delivered: 11,102
• Total opens: 1,488
• Total clicks: 247
• Open rate: 13.40%
• Clickthrough: 2.22%

Group 5

• Subject line: Learn the best days and times to blog, email, tweet and more
• Total delivered: 10,928
• Total opens: 1,414
• Total clicks: 226
• Open rate: 12.94%
• Clickthrough: 2.07%

Analysis

We ran the numbers through the MECLABS Test Validation Tool (part of the MECLABS Test Protocol that our research analysts use with Research Partners). Since the tests weren’t performed in our own labs, we weren’t able to check for possible validity threats and had to assume the data was normally distributed. After making those assumptions, we found that the two best performing treatments showed a statistically significant difference for open rate from the worst-performing treatment at a 95% Confidence Level –  Group 2 and Group 1 outperformed Group 5.

Click to enlarge

Group 2 provided a 9.8% life and Group 1 provided a 7.4% lift over the control.

Why? Well, obviously this is only one test. To truly find the answer would require analyzing HubSpot’s entire testing-optimization cycle But if one were to speculate, these were the only two treatments with a solid promise (of data or a webinar) AND used one of the Five Ws as we call them in journalism (in this case, What and When) to entice the subscriber and let them know that opening this email will begin them on a path that ends with the answer to one of their questions.

That’s just my take. I would love to hear your analysis as well. And here’s what Vreeland had to say about the top performer …

That was the only subject line that opened with “[New Data],” and Vreeland thought that could explain the better result.

He says, “Dan (Zarrella) is known for his interesting analysis of a wide variety of data. It makes sense that those interested in one of Dan’s webinars would be someone who is also excited by the prospect of new data.”

Vreeland adds, “The purpose of the test was to determine the best performing subject line so we could use it on the remainder of our 500,000 email list.”

Each of the top two performing subject lines were used in later email sends during the campaign.

Related Resources

HubSpot

B2B Webinars: How HubSpot drew 25,000 sign-ups, almost 10,000 attendees, and more than 3,500 new leads

New to B2B Webinars? Learn 6 steps for creating an effective webinar strategy

Webinar Promotion that Delivers: Use Email, Social, Viral Referrals and Video to Boost Attendance, Drive Lead Gen

Optimization Summit 2011