Creative effectiveness, modelling and misfiring MMM in the spotlight; Specsavers, REA Group, UM and System1 on why 70% of OOH ads don’t work

From left: JCDecaux's Essie Wake, UM's Raj Gupta, Mi3's Nadia Cameron, REA Group's Sarah Myers, Howatson & Co's Chris Howatson and JCDecaux's Max Eburne
Out-of-home was the fastest-growing media channel in Australia in 2024 surpassing $1.2bn for the first time, according to SMI data last year. It's "loved" by marketers, planners and creatives alike but proving its effectiveness remains a bone of contention if simply reach and frequency provides the benchmark measurement for impact. And that bone only collects more teethmarks as MMM and diverse modelling methods raise questions about what channels really work short and long-term, and billboards continue to be stuffed with too many messages or poor creative that fails to generate brand recall or build memorability. Enter this week’s JCDecaux upfronts events in Melbourne and Sydney, and a fresh swathe of evidence to demonstrate how quality out-of-home environments and creative executions can drive significant brand and business gains through the funnel. Top of the list was global-first research from System1, which found 7 in 10 OOH ads aren’t working, and put forward seven practical principles for doing creative well in response. But there was also plenty of debate from Specsavers and UM on what makes up effective measurement, and how media agencies and marketers embrace MMM and emerging modelling methods to shift the dial from outputs to outcomes – with the hugely important caveat of avoiding garbage data in, garbage data out.
Specsavers: Thinking a channel just does one thing in the funnel is wrong
Specsavers director of marketing planning, Shaun Briggs, says he’s lucky to work for a business that understands both the power of full funnel, plus has a long-term view of what a marketing result might look like. Which has made investing into OOH and going beyond a pure reach and frequency argument possible.
Specsavers recently took over airport billboards across the country with the latest playful iteration of its ‘Should’ve gone to Specsavers’ campaign which saw Sydney arrivals told they were in Melbourne, and vice versa.
“You don't really have to go in and bat for a channel like you might have to in any other place. But I also think the idea a channel does a [single] thing in the funnel is just completely wrong,” Briggs said at JCDecaux’s ‘Be Seen, Be Remembered’ event in Melbourne. “The airport campaign we did recently is a really good example: I guarantee that got us sight tests booked, even though that was not its job at all. Because people just don't walk around thinking in these buckets like that.”
The big tentpole numbers Specsavers ultimately strives for are brand consideration and sight tests bookings – and Briggs knows OOH can deliver both.
“We recognise there’s the short-term – booking a sight test today for an appointment in the next couple of weeks. But then there is [also] that person who is going to fall into category in four or five years’ time. Making sure we're at the 'top of notice' is key. We understand we have to do both [pieces of] work,” Briggs said.
“The ROI is knowing what you're actually measuring, what the return you actually want is. Effectiveness is a word that I think covers heaps of evils – and lots of basic metrics that don't do anything. Effectiveness for me is whether people are considering the brand, it coming into mind at the right time, quickly and easily, and booking sight tests. That is literally it. We're probably good at the creative, yes. But I think we as an industry get tangled up in wanting to put things in neat little boxes that humans just do not fit into.”
The idea a channel does a [single] thing in the funnel is just completely wrong. The airport campaign we did recently is a really good example: I guarantee that got us sight tests booked, even though that was not its job at all. Because people just don't walk around thinking in these buckets like that.
Shifting from outputs to outcomes
For UM chief growth and strategy officer, Raj Gupta, Specsavers is proof of the big shift needed away from optimising media spend against outputs, to planning against outcomes.
“A year ago, we had one client running what we call a full-funnel model, looking at the effect of advertising onto things like consideration – a metric that marketers are keen to understand – then looking at the effect of that consideration onto a sale. But that impact might not be next quarter, it might be in 10 months’ time. Or in Specsavers’ case, it might be in many years’ time when someone needs glasses,” Gupta told attendees. “You've got to be able to measure that lag effect to see whether there is some effectiveness. Because if consideration in the long run doesn't help you, why are you measuring it?
“This shift from an output-based world in which we're optimising things is very different to a shift to an outcomes-based model. I think people need to start thinking of those outcomes that are right for our clients’ businesses, whether it's short-term, immediate sales, or whether it's long term in terms of those attitudinal metrics, or from a behavioural point of view. But those outcomes are really what I'm starting to see our clients focusing on.”
According to Gupta, UM now has 17 clients moving in the direction of full-funnel measurement, with key metrics determined by the type of business they're in and what they’re trying to achieve.
In one test with a client running a full-funnel model, UM looked at what an attentive reach optimised plan could deliver. That plan delivered 84 per cent weekly reach for that client.
“The media mix looked like what people might say a traditional mix looks like – it had television screens, a bit of out-of-home in there – probably not enough out-of-home, I'll be very frank about that – plus a lot of digital,” Gupta explained. “We then re-optimised it, based on that full-flow model. We wanted a balance in terms of wanting to see a growth in consideration over three months but also wanting to see its sales effectiveness. It dramatically changed the mix. Out-of-home came up within that mix; certain parts of digital that we might attribute to last-click attribution came down. We then ran the reach numbers – you have to give marketers comfort in what you’re doing and that you’re not just giving everything away without that reach number being there.
“We sacrificed about 7 points of reach – from 83 down to 76 – and increased ROI in the short-term from 11 per cent on marketing and media up to 76 per cent. For a 7 per cent reach reduction, but higher quality in what you're doing – and I will say quality does matter, we've seen the ROI effect – to then generate 65 per cent additional ROI? I don't know a marketer who wouldn't take that.”
It's one thing planning to this approach; it’s quite another to make sure you’re holding yourself accountable to the result as well, Gupta added. “We then improved consideration after the three-month period.”
Getting a full-funnel, nuanced view of all channels in aggregate through the funnel is also something REA Group’s GM of marketing and audience, Sarah Myers, says her team has been refining for the last few years.
“When we had static assets in outdoor and 100 per cent share of voice, it was about frequency. But as that evolves and platforms digitise, the onus is on us marketers to determine the effectiveness,” Myers said on the Sydney leg of the panel. “You had CEOs and CFOs not necessarily seeing certain assets when they were going past. So we had to approach it differently.
“For many years now at REA, we have worked with Kantar and do deep studies – we know for every asset and channel, what role it plays in our marketing mix. We also know how it works for us, through the funnel, and what it does. We look at share of spend and share of brand impact, and especially at the top funnel, we see twice the brand impact than what we see in terms of spend [in out-of-home]. But that works down the funnel. Preference is success for us in our category, and we’ve really accelerated that. And we know outdoor is playing a big role in that," she said.
“We have been finessing the modelling over the last four to five years, and there are different combinations across street furniture, transit – they do play different roles at different times and in different markets. We have seen real nuances between Sydney and Melbourne in particular.”
These aren’t the norm, however. As JCDecaux CMO, Essie Wake put it, most marketers and planners love out-of-home, but its track record on demonstrating results is “patchy at best”. The lack of demonstrable one-to-one connection, click through rates, traceable pixels, a lack of seamless integration with attribution models, plus sole reliance on legacy metrics such as reach and frequency have all contributed to the challenges of proving OOH’s worth, she said.
“Out-of-home has an accountability gap,” Wake told attendees. “If we start with the basics, look at what we’re all measuring: Are we measuring different aspects of the picture without seeing it’s full effect or really getting to the heart of what makes out-of-home work? We’re deploying media mix modelling without quality exposure data potentially, and we’re measuring sales uplift and profitability when the brief objective to us might have been for awareness and consideration.”
We look at share of spend and share of brand impact, and especially at the top funnel, we see twice the brand impact than what we see in terms of spend [in out-of-home]. But that works down the funnel. Preference is success for us in our category, and we’ve really accelerated that.
The MMM effect
While the growing instances of Market Mix Models (MMM) being implemented are designed to help overcome clunkier attribution models and help brands optimise media holistically to gain more effectiveness, UM's Gupta expressed concern in their role as a short-term measure to sales.
Given the threat MMM potentially poses to agency planning strategists, it’s no surprise questions are being raised around MMM agency-side. Gupta wasn’t the only one questioning MMMs at the JCDecaux event, though. While reserving its arsenal for broader and deep conversations in the longer term, JCDecaux did flag questions around whether OOH’s priming effect and the nuances of consumer journeys through public spaces are being fed into the models in a way that adequately reflects their impact on brand and business outcomes.
Alongside Gupta, JCDecaux head of insights and market data, Cristina Smart, also highlighted the dangers of “crap in, crap out” – whether within creative, modelling or measurement.
“If you don't get the right inputs going in, that’s what happens. This is my issue right now with MMM and out-of-home: I don't think we're putting the right measures in to properly attribute the sales out-of-home is generating,” Gupta said. “My example for you: If I put per $1 million into the first of the month into my system and try to attribute it on that day, rather than the fact that that million dollars might be spread out over 30 days, so in effect it’s $30,000 a day, you're going to get the wrong measurement.”
With so much to still learn around MMM, modelling and data, Smart encouraged clients to engage in an open discussion about OOH’s performance.
“Whether it’s MMM, brand tracking, sales data or post-campaign analysis, what matters is what we question collectively together,” she said. "Maybe some of the things we need to question are: What was the real out-of-home audience exposure? How can we integrate geography into a model? What’s the true out-of-home impact in the consumer journey? What’s the priming effect we’re trying to measure? Are we looking at the right outcomes collectively, and do they correctly correlate to the initial brief we were receiving? And the one I’m most passionate about: Did we supply you with the right data to make sure we measure the impact as close to real life as possible?”
We sacrificed about 7 points of reach – from 83 down to 76 – and increased ROI in the short-term from 11 per cent on marketing and media up to 76 per cent. For a 7 per cent reach reduction, but higher quality in what you're doing – and I will say quality does matter, we've seen the ROI effect – to then generate 65 per cent additional ROI? I don't know a marketer who wouldn't take that.
Global-first creative effectiveness principles for OOH
Outside of measurement effectiveness, a major focus of the JCDecaux event was global-first creative effectiveness research and evidence-based creative principles presented by System1 SVP of global partnerships, Andrew Tindall. The pragmatic steps on how to lift the impact of out-of-home efforts creatively came out of testing 87 Australian out-of-home campaigns with more than 13,000 Australians nationally to determine the impact creative quality has on business and brand outcomes.
System1’s new research found seven out of 10 out-of-home ads are not working, because 50 per cent fail to land brand recognition within the two seconds of attention consumers are generally accepted to put towards an out-of-home exposure. The figure for Australia was slightly higher – 60 per cent against the global 52 per cent – yet the findings still reflected the significant opportunity that exists to make OOH work harder for brands, Tindall said.
What’s more, half of those brands actually recognised don’t create emotional engagement that leads to changing behaviour. Those that do score higher on System1’s creative star rating for creative testing – 3 and 4 out of 5 – achieve double the commercial effect when compared with years of commercial ROI results recorded by JCDecaux in its client campaign database, Tindall said.
Off the back of the research, Tindall outlined seven key principles for OOH creative:
- Brand fast – because you only get two seconds of attention.
- Don’t be dull – Dullness is your brand’s biggest enemy and costs your efficiency. The top 20 per cent happiest campaigns achieve +167 per cent more key brand association lift than the dullest 20 per cent of campaigns.
- Stay consistent – Focusing on critical brand assets and staying consistent helps compound impact.
- Attention changes minds – While dwell time greater than 2 seconds don’t increase brand recall, it has a significant positive impact on brand preference.
- Less is more – OOH ads with more than 1 message saw material decline in brand association.
- Make them feel good – positive emotions drive results.
- Showmanship over salesmanship – Right-brained creative using things like character-based fluent brand devices, humans and dramatic scenes drive engagement.
As the epitome of Tindall’s point about both not being dull and creative consistency compounding impact, Briggs pointed out Specsavers sells glasses and therefore has to make itself interesting and memorable.
“If we were dull, that would be a disaster. Because we sell glasses, we have to do things that are interesting ... We have to have a bit of fun, carefully,” Briggs said. It also needs to remain consistent.
“We have been clever enough to stick with what works and keep doubling down on that without any of the science – ‘Should’ve’ was born 25 years ago, but we've just stuck with it because it works and now there's all this evidence there to prove it does. We're true to that and what we’re about.”
Being more selective with your channel mix
Off the back of the insights, Howatson & Co founder and chief, Chris Howatson, suggested it's time to completely rethink out-of-home's role in the mix.
"Most of us were probably brought up understanding out-of-home as the follow-up channel. The key outcome for today is it's very much the lead channel for what I would imagine is a massive chunk of the market that can't effectively spend across channel," Howatson said.
"Advertising, at its the most basic, is incredibly simple: We're priming someone so that when they come into a buy moment within that category they're going to think of us as first or second in that kind of room. That's it. That's all we have to do – that is the fundamental basis of advertising.
"If you're a brand that doesn't have any distinctive codes, you're not going to achieve necessarily those knockout metrics on the first round. Or if your brand that doesn't have enough budget, you're going to end up prioritising reach at the consequence of frequency, or you'll just have a whole lot of music out there that doesn't create impact."
Howatson's other big call in the face of channel proliferation and ever-tighter budgets? Get more selective and strategic about your channels.
"I think in the past we've had enough money to buy everything. Now, we don't. Maybe that's the transition moment in the range – we don't think about the classic plan as screens, then out-of-home. Perhaps it's about owning a share of voice within a channel. If you can't ensure a voice across channels, pick one.
'I remember the great case study about Pepsi, which could never maintain its share of voice against Coke. So they said we're going to own out-of-home and that's that. Later on they went into digital and I think became a little bit lost since then as they're just not as present as they used to be. And then look at White Fox, a $100 million retail business that's built itself on TikTok and out-of-home transit," said Howatson.
"Reductionism essentially is maybe the next conversation we go to."