Data Doesn’t Lie, but Your ROI Metrics May Mislead

Marketing leadership likes (and frequently trusts) real-time reports including ROI-like metrics such as ROAS. These constantly updated reports always provide something new to look at and promise opportunities for advertising tweaks. The real-time nature of these types of reports can be compelling. But this creates a big vulnerability: Every player in an organization has their own siloed metric telling their story (“their story” not “THE story”) about their partial contribution to the business. Real-time reporting on ROAS provides instant gratification, but often fails to tell the whole story and often contradicts the reality of ad effectiveness.

Where siloed metrics fail is in delivering a unified and accurate view of marketing and its impact on business performance. Siloed metrics like ROAS (Return on Ad Spend), provided by platforms such as Google Analytics and Adobe Analytics, and by some brands and agencies directly, and digital click metrics by channel do not provide a true picture on performance. These can be valuable insights into individual ads and campaigns, but measuring marketing performance and brand impact by adding up siloed measures is guaranteed to be misleading. If both Google and Facebook contributed to a purchase, it is guaranteed that the siloed reporting from both ad platforms will take credit for the sale. Adding up siloed reporting will always over-report performance.

Last click and other user-level metrics fail to account for other touchpoints in a customer’s journey that contribute to the sale or other KPI. For instance, they might incorrectly attribute clicks and impacts to paid search, ignoring the fact that a large portion of search clicks are generated by other types of marketing. The Person A example below illustrates this.

Additionally, search metrics often claim credit for sales that would have occurred regardless of whether a marketing campaign was run or not. Web analytics data shows that search overstates its impact by two to ten times the amount.

To illustrate, suppose a company relies solely on siloed measurements to evaluate its marketing performance. They notice a wide range of ROAS by tactic, ranging from $0.01 to $22, and decide to shift their spending towards affiliates and paid search, assuming that these channels drive the highest ROI. However, this narrow last click view fails to consider the full picture. For example, video drives sales despite missing clicks, has a longer half-life than non-video media and can drive up to 30% of search clicks. As a result, using ROAS alone can lead to incorrect conclusions about which channels are truly driving sales. In fact, in some cases, it may assume that 100% of sales are due to a particular channel when only a small percentage are.

Many marketers make investment decisions using ROAS data. Depending on how siloed metrics are shared throughout the organization, there is a significant risk of misinterpretation. As companies emphasize quick decisions, it can be easy for business leaders to forget what these real-time reports with siloed metrics actually represent and that they do not tell the full story about marketing performance.

In a piece titled “A media effectiveness guide for CMOs (and CFOs),” Google also agrees that siloed metrics like ROAS have their limitations, saying:“Measuring sales impact may require using more than one tool, including offline and online media as well as offline sales channels, to ensure comprehensiveness and granularity.”

It’s time to take stock and reassess the utility of the deluge of data and analytics we have access to every day. Real-time reports should be leveraged within the silo they represent. Businesses should interrogate whether these siloed metrics and real-time reports are being applied appropriately. Are we looking at the whole forest, or are we fixating on a small section of trees?

To avoid the pitfall of a siloed approach, start with a full business perspective and customer-centric measurement approach. With a commercial view, where the customer is at the center, you can better understand what is driving business performance. A privacy-safe, customer-centric perspective showcases the role of marketing in the context of the full commercial business drivers – brand, product, operational, and external factors.

Shifting to a commercial view avoids costly, blinkered decisions and helps cut through biases. As has been widely reported, the prevalence of real-time reports can create a bias that comes at great cost.

Everyone in your organization has the same goal: To add value and grow the business. But many participants only have a siloed view on marketing and brand performance. The siloed views are often valuable within the silo from which the data and metrics originated. However, a best-in-class marketing organization relies on a customer-centric commercial view to make superior decisions that have short-term, long-term, and brand-building impact. It is imperative for marketers and organizations to ensure that real-time reports do not misinform and cause poor investment decisions. Data doesn’t lie but it can mislead.