Monday, October 1, 2007
Consider randomized, real-time testing
I think all marketers have had a similar experience: You're meeting with a client/superior/colleague discussing an upcoming campaign. Your client is absolutely sure that one offer/creative will outperform another. You, on the other hand, just KNOW that the alternative offer/creative is the winning package. You argue about the benefits of one over the other. Voices are raised; you're passionate; YOUR idea is the best. What happens at the end? Typically, the debater with the most clout (i.e.: the client or the boss) wins. Their idea ends up getting implemented...
What's the better resolution to this dilemma (and the answer that direct marketers are typically first to suggest)? It's simple--test both concepts and let results decide the winning idea. Unfortunately, however, rigorous testing just isn't done often enough.
This article from Advertising Age, Randomized Testing Is Fast and Cheap, but Few Seem Interested made me think about testing in a new way.
It's all about real-time testing and the author cites an insightful case study of how an author let clicks from a Google ad determine the title of his soon-to-be-published book. The author was certain that he had the right title. His editor was certain that another title would be more interesting. Well, the editor was right, based on responses to the Google ad.
The article cites other testing examples and also discusses how the Internet has enabled real-time testing.
It may be time to evaluate your testing strategy. RRW has written a white paper on this subject. If you'd like a copy of that white paper, just comment on this blog and we'll send it your way. We've also discussed testing on this blog a couple of months ago. Testing is always a good idea.