Oscar Wilde once wrote: "The only thing worse than being talked about is not being talked about."
Marketing mix modeling has been getting a lot of press lately, and deservedly so. As the larger business community has begun to comprehend the challenges and opportunities presented by "big data," many have rightly observed that the marketing research industry has been benefitting from big data for decades now.
Of course, with publicity comes inquiry and criticism. As long-term practitioners and leaders in marketing mix modeling, we welcome any open discussion about research techniques to quantify and improve marketing ROI. Therefore, we welcome the April 24, 2013 story in Ad Age entitled“Marketing-Mix Models get Pushback As Media Landscape Changes.”
The Ad Age article explains:
Some critics believe the models have been wrong all along, and work even worse after three decades of change in the media landscape. They say the models underestimate the impact of advertising, particularly of broad-reach network TV; overstate the value of price promotion, mislead marketers into buying thinly rated programming; wrongly downplay risks of going dark for weeks on end; and fail to account for how online search has made all advertising more effective.
While it is true that some modeling practitioners have not evolved with the times, and in some cases they may have constructed poor models from the start, the sentiment that marketing mix models, as a whole, are inaccurate is overbroad. Marketing mix models have become commonplace in businesses despite a significant amount of evaluation and interrogation over the years simply because they do, in fact, work. In our experience, marketing mix models predict well, they forecast well, and the proof quickly becomes self-evident as real sales performance either contradicts or validates the model predictions.
Marketing mix models measure what drives sales for a business. By leveraging an enormous amount of data and sales trends by week, and by geographic location, this type of research allows companies to understand the role that each individual marketing tactic plays in driving sales. Also marketing mix models help marketers understand how tactics work in conjunction with each other and other business and economic factors in a holistic way. What’s important to note is that models are constructed on actual sales data (as opposed to a sub-set of sales performance or self-reported consumption or opinion data), thereby capturing the real ebb and flow of how the business performs in the marketplace.
Not all models are created equal, however. There are different types of methodologies that may be leveraged, and different levels at which sales trends can be measured with this type of research. The criticism that some models “overstate the value of price promotion” is a valid observation of some modeling techniques. For example, we have long observed that marketing mix models that are constructed at the store-level tend to overstate the value of promotions and thereby understate the value of media. That is not to say store-level models are never called for, but we find in most cases, market-level models are more accurate and do not have a bias toward price promotions.
Our philosophy is that models must be constructed at the level at which business decisions are made. When it comes to marketing mix models, the most accurate level at which to measure media while controlling for promotional activity is at the market (DMA) level where media is typically purchased and delivered. While there are a number of issues with marketing mix models that are constructed at the store-level, it should be noted that there is also a great deal of value in leveraging store-level data for certain types of business questions. Contrary to the criticisms in the Ad Age article, we often recommend increasing media investments while decreasing promotional investments – particularly those focused on price discounts. See our recent article cautioning about promotions and discounting entitled Tips for Managing Your Company's Promotional Strategy.
Analytic Partners results are, in general, counter to those noted and criticized in the Ad Age article. Our results often suggest that advertising warrants additional investment, and our findings often recommend continuity with advertising and avoiding off-air periods (as long as budgets allow, of course). As for Online Search, well, that can work very well for many businesses but results vary by industry, product, and specific circumstances.
The key to leveraging marketing mix modeling – and any research for that matter – is to understand the role that this research should play in making business decisions. High quality marketing mix models are accurate, but like any research, the results should be used as an aid to judgment. Analytic Partners continues to emphasize that the value of this research comes not from the models themselves but from the insights our clients gain by evaluating and testing results to learn more about the drivers of their business. Marketing mix models are an aid to judgment for marketers, not a replacement for good judgment from experienced marketers. We also believe that some marketing mix vendors have wrongly tried to commoditize marketing mix models, and we speak to these points on our website. Please visit the following link and click on the “Perspective” tab for more information.
Analytic Partners would welcome your questions and comments. As with all of our client engagements, we believe transparency is crucial to the success of this type of work.