One of my favorite aspects of working in a family of research companies is getting to see new theories tested. It is a core essence of our “Discover what works” tagline, and when a recent “discovery” experiment came across my desk the other day, I couldn’t help but want to share its implications with our audience. Let me explain the background and give you a little preview of the discovery that will be fully revealed in today’s web clinic.
What’s the story?
We are currently running tests with a large subscription-based news service that has a large direct mail (DM) aspect to their marketing campaigns. We saw this as an opportunity to test some of the hypotheses we have been formulating related to the connection between offline and online efforts.
Now, as with most DM efforts, their campaigns had a call-to-action that included an online option. For those interested in signing up for the offer online, they could go to a specific URL. The URL was unique, the path was clear, and the impact of the direct mail campaign would be easy to track. Right? Well…
Those DM rebels
We wondered how many DM recipients were wandering off the beaten path of a nice clean and trackable vanity URL. We questioned how many people who received the offline campaign were rebelling against the directions and forging their own way to what they wanted. And overall, how much of this company’s generic website traffic was taking part in this underground rebellion?
All these questions led us to test the notion that DM pieces might be having a greater impact than planned for and that we might be underestimating the significant opportunity created by our offline efforts
How we set up the test
So we set up a split test that would integrate some of the messaging from the offline DM campaigns into a generic, but similar product offer page on their website. The elements we tested initially were mainly images. The DM pieces relied on strong images and, to create continuity, we used similar images on the generic website pages (see below). This test was also specifically set to run during the time that the direct mail piece would be landing in mail boxes. (out of courtesy to the Research Partner, we have anonymized these pieces):
Original Offer Page: Direct Mail Offer: New Offer Page:
Our thought here was that if there was any traffic coming from the direct mail to these generic offer pages, that the images would increase relevance and have a positive impact on conversion if many people were coming from the DM campaigns. We were hoping to see a difference in the results over time related to the offline campaign.
The Results – A 124% increase in subscriptions
The results were stunning. During the two weeks that the DM was out, we saw a 124% lift in subscriptions for the new page. After the two weeks, however, the conversion rates for the page that integrated imagery from the DM campaign went back down to its historical average.
This supported our hypothesis that offline efforts have a larger impact than might be expected and that people don’t necessarily follow the prescribed path of a DM campaign. For this company, many ended up on generic product pages.
What does this test mean for all of us? I believe we can learn two things: 1) If you are running offline campaigns, be aware that they may be having a larger impact than you think and 2) Identifying where response to your offline campaigns overlaps with your online visitor’s engagement, and establishing a connection between the two messages can generate significant response.
For this company, discovering that overlap and connecting the messages meant 124% more subscriptions. What might it mean for the rest of us?
For a deeper explanation and analysis of this experiment, activate your FREE MarketingExperiments email subscription to be notified when the replay of this web clinic is available.
Related Resources
Thank you for this. I am few weeks away doing online/offline test myself and I needed to get my thinking wheels in motion. Awesome post and I am glad it worked for you well!