During World War II, Allied bomber planes faced a critical design problem. They were slow, lumbering and constantly shot down. The bombers needed to be reinforced with armor, but covering an entire plane made it too heavy to successfully fly a mission. Reinforcing them only in their most vulnerable areas, however, could solve this challenge.
Naval engineers mapped the location of bullet holes that had struck the returning planes to reinforce areas most often hit. Statistician Abraham Wald noted that this analysis only looked at planes that survived since it was impossible to assess damage on planes that were shot down. Wald concluded that the areas hit by bullets weren’t vulnerable – these were areas where the planes could get hit and still return safely. Instead, they needed to secure areas where returning planes hadn’t been shot since those were more likely to fail if hit.
The naval engineers’ original plan to only reinforce the bullet-ridden areas is called survivorship bias. This sort of flawed thinking leads to false conclusions and misguided decision-making by only looking at people or things that survived some event or process, while overlooking those that did not.
In digital marketing, a similar mistake is being made with attribution. The current state of attribution is flawed because it only looks at completed campaigns to measure conversions and identify top performers. Traditional attribution models, such as last-click, first-click or multitouch, falsely assume that for a conversion to take place, an ad must be shown. This reasoning fails to acknowledge that certain conversions would have taken place regardless of ad exposure, and it causes even the savviest digital marketers to unjustly reward specific media partners and inflate ROAS.
Consider, for example, a campaign that is designed and executed to maximize post-click conversions. A sophisticated media-buying platform powered by machine learning is trained to find and target the lowest-hanging fruit – users most likely to convert – to ensure campaign goals were hit. In this case, the platform gets credit for conversion.
In truth, many of those users would have converted without seeing an ad, perhaps because of seasonality or brand loyalty. But an intelligent system finds and targets these users excessively because it knows they have a high likelihood to convert.
Should marketers spend money to reach those people? Should revenue from those users be taken into account when evaluating ROAS? Or would marketers be better off redirecting those dollars to reach people who truly need to see their ad to convert?
There may even be some cases where users who would normally convert on their own would be less likely to convert after seeing an ad, potentially the result of misaligned creative or overexposure. But because traditional attribution models only measure and reward performance against impressions shown, the related campaign might still be deemed a success so long as it hit its CPA benchmarks.
This is obviously (and thankfully) not the case with every campaign and optimization strategy, or we would all be out of work. But, it does highlight the critical need for brands to rethink what they know about their audiences and how they plan and evaluate digital buys.
I get it: The notion that ad exposure is not always needed to drive business outcomes seems radical, especially for advertisers. Digital-first companies, like Netflix, are already adopting sophisticated technologies that enable them to more efficiently find and target users that drive incremental value, rather those that would convert regardless. And traditional brands like Allen Edmonds are starting to question the real incremental value of their retargeting activities and reallocate those dollars to more efficient channels.
These companies have upended the way they think about digital advertising to gain greater clarity on ROAS and drive greater value for their businesses. They’re reinforcing their planes to ensure future survival.
Originally see on “Data-Driven Thinking” via AdExchanger.com.