With legislation like the California Consumer Privacy act and platform policies limiting the availability of identity data, it has never been more important to have an accurate understanding of the biases and risks associated with data loss. More importantly, brands need to understand how to best combat those biases.
At a recent ARF event, Analytic Partners shared our results from a series of experiments designed to better assess the impact of data loss based on business outcomes from measurement. In performing this experiment, we leveraged a Unified Modeling approach to measurement which included integrating Commercial Mix Models (CMM) with Multi-touch Attribution (MTA) modeling in an iterative process. The process synchronizes the model insights from each technique through proprietary algorithms that are applied for speedy convergence.
To combat traditional MTA challenges such as walled gardens, blind spots, seasonality, and data quality issues, Analytic Partners utilizes an innovative attribution modeling approach that estimates the impact of marketing activities on conversions, while simultaneously adding clarity.
We performed simulated data loss experiments within two different scenarios:
Effects of Data Loss by Methodology Type
The graphs below represent the effects of data loss by methodology both when the data is randomized (scenario 1) and when it’s specific (scenario 2).
Results remain relatively stable, with a Unified Model performing best, until the point of 50% data loss, when results degrade regardless of methodology.
Specific data loss (by demographic, income) has a greater impact on outcomes than randomized data loss.
Across scenarios, a Unified Model outperformed MTA-only and Last Click methodologies. We’re able to clearly see that specific data loss can have a major impact on outcomes relative to randomized loss. These experiments demonstrate that:
In addition to the results and insights above, we also tested the impact of data loss on digital display rankings in scenario 2. The first graph demonstrates rank order for digital display types post-data loss utilizing only the Unified Model approach, while the second graph demonstrates the differences in outcome across all three methodologies pre-data loss.
While the majority of best and worst-performing display types remain similar, meaningful differences in rank occur post data loss.
There are significantly greater differences in outcomes based on the methodology used versus simulated data loss.
The key takeaway from the results seen in these scenarios is that oftentimes methodology, more than data loss, affects outcomes. With that in mind, we recommend that brands consider the following when handling similar challenges in data loss:
The challenges associated with data loss are inevitable in our constantly changing landscape. Disruption is the new normal, but with the right tools and partners in place – in tandem with a holistic view of their business – brands can adapt to new environments and navigate these challenges with confidence.
Analytic Partners would like to thank the ARF Cross-Platform Measurement Council Attribution Working Group for providing the original parameters of these experiments. These results originally shared at ARF DataXScience 2019.
- Preeti Croke, Senior Director at Analytic Partners
Analytic Partners can help your business adapt.Contact Us