Email Testing: Subject line increases opens 12.7% … here’s why we’ll never use it again
The purpose of a subject line is to get an open. However, the purpose of a subject line is not only to get an open. In our recent subject line contest, there were some curious submissions that made me think this blog post was necessary:
- Mom told me to wear clean underwear in case I was in an accident. I wish she had told me about this too …
- A priest, a rabbi and a MECLABS LPO expert walk into a bar in Denver …
- RE: The video showed a bald man. Why?
- 1 Thing You MUST Do in Denver Before You Die
- Want to find the end of a rainbow AND the pot of gold?
- Denver Flight #ME430 [Confirm your seat]
- Open &; Enjoy Real Bacon Smell…
- This is your brain on email.
- What the #@*? What should you test next?
I know some of these are likely meant as jokes, but if the amount of misleading subject lines that fill my own inbox every day is any indication, many of them are probably serious.
The road to unsubscribes is paved by good intentions
And I don’t mean to pick on our contest entrants. If fact, I’m going to show you a subject line test I ran making this same mistake later in this post. Even in the blog post, “Announcing the Winner of the Email Subject Line Copywriting Contest!” by Sonia Simone, CMO, Copyblogger Media, she discusses how I pushed back on one of her selections because I was worried it was misleading …
For example, we really liked the header “Testing — does this link work for you?” (We defined that one as a Curiosity headline.) But MECLABS had some concerns that the element of trickery would annoy their subscribers and lead to unsubscribes … definitely not the result we were after.
So why do we, as marketers, come up with these misleading subject lines? Well, I think our intentions are in the correct place: We’re simply looking to stand out in a crowded marketplace. However …
Email is a circle of trust
Keep in mind that by having someone’s email address and the permission to send them email, you are entering into a circle of trust in that person’s life.
Sounds kind of touchy feely, doesn’t it? But think about it. The inbox is where people share some of the most important things in life with the people that matter most to them – from important results at work to moments with the family or friends at home.
If you mislead, you violate that trust, and your audience will fight back by unsubscribing, or, perhaps even worse, labeling you as a spammer.
… and in the end, curiosity doesn’t work very well anyway
Sonia puts these types of subject lines into the greater “Curiosity” category. And while there are ways to raise curiosity without being misleading, the curiosity subject line that we ended up going with for the recent subject line contest – Quarterbacks aren’t the only changes being tested in Denver. – was the worst performer of the six subject lines we tested.
“Curiosity headlines rarely do well. Writers love them, but readers tend not to respond. That’s not something we invented or discovered; serious copywriters have been preaching it for more than 100 years now,” Sonia said.
“We’ve seen it over and over again when one of us falls in love with a clever headline idea.”
Of course, to understand what really works, you must have your eye on the correct KPI, and as I began this blog post by saying, that may not be the open rate when you’re testing a subject line.
In a recent case study email sent to the MarketingSherpa Email Marketing Newsletter list, I fell in love with a clever subject line written by our freelance reporter, Jeri Dube (note: MarketingExperiments and MarketingSherpa are both owned by MECLABS).
Today’s test is not unrelated to this milieu …
I thought, “What a perfect chance to run a test that challenges the model.”
The MarketingSherpa subject line model is pretty straightforward – we write case studies and how-to articles that focus on “what really works” in marketing, and write straightforward subject lines that reflect the results from those articles when we distribute them.
Control: How Blockbuster Express grew its email list 300%
Treatment: If Alfred Hitchcock wrote emails, could he grow an audience by 300%?
It was the perfect crime. Two unrelated subject lines meet on a train …
Sorry, that curiosity-driven subject line really got me going. And the audience, too. It generated 12.7% more opens.
But this test has a Hitchcockian twist. Even though the subject line had a higher unique open rate, it had a lower unique clickthrough rate (measured as delivered to click).
So even though the Treatment had a head start (if more people open it, there is the possibility for more people to click), it received less unique clicks.
The Control made an honest promise: “If you open this email, you will learn more about this case study.”
The Treatment was basically just saying: “Hey, dude, wanna see something cool?”
That is why I’m sticking to the tried-and-true model for the MarketingSherpa editorial subject lines. It’s always good to challenge that model, but it’s essential to focus on your true KPI. I don’t care how many opens we get if we make a misleading proposition that discourages the marketer from reading the article.