Skip to main content
Deep Dive 17 Jul 2023 - 10 min read

‘No hallucinations’: NAB, Tabcorp, Accenture Song on 'applied AI’ - less ‘possible', more ‘pragmatic’ use cases first; NAB flags AI creative surge for 500m personalised messages but brands warned on mindless messaging, CX usurping media for brand building

By Paul McIntyre & Brendan Coyne

Pump up the volume – and the creative quality – or fail. L to R: Accenture Song ANZ President, Mark Green; Tabcorp Data & Analytics Chief, Amy Shi-Nash; Accenture Song Creative Chairperson, Nick Law; ANZ CMO, Suzana Ristevski.

NAB CMO Suzana Ristevski is a touch emphatic after just completing a three-year, $45m grinding overhaul of the bank's marketing technology systems that has seen 95 per cent of the tools used by the marketing team switched out – a Pega decisioning engine and Tealium's customer data platform are among the new line-up. Now NAB is ready for the AI stuff.  Ristevski acknowledges “inflicting a lot of pain” on the bank’s marketing team in the tech transformation and there’s more to come but it's "more exciting" as NAB works to embed machine learning and generative AI to deliver personalisation at scale that isn’t just “sending out rubbish”. She’s already sent out 10x more comms, now circling at 500m. But the bank is acutely aware of the risks posed by AI – and the strategy goes all the way up to the top. Tabcorp is onto its third in-house generative AI program and the creation of a centralised unit, Next Labs, to experiment – the problem with the other two was that they lied, or suffered “hallucinations”, per Chief Data & Analytics Officer Amy Shi-Nash. But Tabcorp can’t risk plugging into things like ChatGPT. The Monkeys and Accenture Song ANZ boss Mark Green, meanwhile, is back from Cannes where the AI banter was redlining on a "moral crisis" crisis for creativity, not dissimilar to the previous metaverse hype cycle. Green’s not worried. He reckons he’s slipped Accenture Song Global Creative Chair Nick Law an idea “that’s as good as any we’ve ever created” with generative AI at its core – and he says the client briefs are coming in from all angles. New York-based Law thinks there are new AI risks which will undermine the fascination and effectiveness of personalisation – namely, tonnes of mediocre messaging that will be created under the guise of 'right person, right time', unleashing a sea of sameness. NAB's Ristevski agrees.

What you need to know:

  • NAB has completed a $45m, three-year overhaul of marketing technology, switching out 95 per cent of its ‘legacy’ tech tools, per CMO Suzana Ristevski.
  • The grail is personalisation at scale – and NAB has already driven 10x messaging growth since the overhaul commenced, moving from 50m messages to circa 500m.
  • AI – the marketing function is driving the early experimentation and use cases at NAB – is being backed to help deliver better personalisation, but Ristevski’s acutely aware of the risk of pumping out “rubbish”.
  • Accenture Song global creative chief Nick Law agrees. He thinks the industry’s last wave of disruption – programmatic media and targeting – missed a massive opportunity by obsessing on the pipes and wires and forgetting the creative and strategic rigour.
  • AI-driven personalisation risks making the same mistakes. “You can do as much personalised messaging as you want – if the quality is not good, it doesn't matter,” per Law.
  • Tabcorp is also moving on AI, building out a centralised ‘Next Labs’ unit to drive experimentation and use cases. The wagering firm can’t risk plugging in to ChatGPT and is now on v3 of its self-built large language model.
  • The problem? “Hallucinations”, per data and analytics chief Amy Shi-Nash. I.e. it lies, which isn’t helpful in a marketplace where accuracy is paramount.
  • Meanwhile, as debate swirls on whether CX now delivers brand building better than ads, NAB’s Risevski is emphatic. “That’s rubbish,” she says, because it only really works for existing customers. “We still need to bring people in.”
  • Plus, as the creative community descends deeper into “moral panic” about AI’s business model disruption, Accenture Song’s Law and ANZ boss Mark Green think the fear factor is overcooked. For now, AI-generated creative remains “a parlour trick”, says Law, and will just make the good stuff more valuable.
  • But Green says ideas with AI at their core are already moving the needle – he’s working on AI-powered ideas that are “as good as anything we’ve ever created”.

We've created a technology stack that gives you scale. Now is the perfect time to think about how you add the creative to that. It's one thing to create the right decision at the right time through the right channel. But I'm excited because it's also [about] the right creative. Otherwise we're sending out rubbish.

Suzana Ristevski, CMO, NAB

NAB CMO Suzana Ristevski is emerging from three-year martech overhaul. “I inflicted a lot of pain on the marketing community as we pretty much swapped out all the tools,” she admits. Now she’s about to go again, this time working out how to embed machine learning and generative AI into the business to help deliver personalisation at scale that isn’t just “sending out rubbish” to the bank’s 8.5 million customers.

The $45 million tech overhaul – $15m a year for three years – has seen NAB “get rid of what we call legacy technology … because it wasn’t fit for purpose”.

NAB, like rival Commbank has opted to bring in Pega Systems for decisioning. Meanwhile, it picked Tealium for its CDP over the likes of Adobe and Salesforce.

“We wanted to create ‘one way, same way’ in our ability to do marketing communications. That sounds simple, but for a bank, it's actually quite complex, because we are piped into a number of different platforms. So it's been an incredibly complex process,” she says.

Having been through the pain, “we are [now] able to do personalised communications at scale,” says Ristevski. Plus, the marketing team is plugged in to the same system: “One tech stack, one data decision engine, one campaign system, one data platform.”

There’s still some way to go: “We haven't quite got off a couple of systems that we want to get off,” Ristevski admits. But NAB has been upgrading on the fly while ramping up output. The upshot is that the bank has increased its personalised comms by an order of magnitude since commencing the rebuild. It's also already taking a pragmatic approach to AI-enhanced customer service using chatbots. 

“Machine learning and artificial intelligence has been around for quite some time and the way we've approached it is we have to run and change at the same time. So we didn't stop for three years, we had to continue to do effective and efficient one-to-one communication. So we've been doing that and we've been adding use cases, adding campaigns to it,” says Ristevski. “We were at 50 million personalised communications three years ago. Now we're now circling around 500 million personal communications.”

Believe the hype

Ristevski doesn’t think AI’s potential is being overhyped.

“I don't think we're overcooking it …  I think our challenge is going to be how do we integrate it into the technology that we've already got?”

But that’s no different to bringing in any new technology, says Ristevski.

“It’s the same as when we were rolling out our existing marketing technology stack. We need to think about how we integrate it into our existing technology, rather than creating Frankenstein technology and data management. That is a critical success factor and will continue to be a critical success factor. What goes in is going to determine what goes out.”

Failure to link all that intelligence to the messaging undermines the whole thing, per Ristevski.

“We've created a technology stack that gives you scale. Now is the perfect time to think about how you add the creative to that. It's one thing to create the right decision at the right time through the right channel. But I'm excited because it's also [about] the right creative. Otherwise we're sending out rubbish.”

[AI-driven personalised comms at scale] have to be aligned to the broader brand message, the broader emotional priming, the what is it that my brand represents? And that's the challenge. It's as simple and as complicated as that.

Suzana Ristevski, CMO, NAB

Through-funnel intelligence

Equally, Ristevski sees linking the top and bottom of the funnel as a key challenge that generative AI isn’t going solve it by itself.

“The more digital we get, the more human we need to be. So firstly, you can never replace creativity. I think the trick is, how do you use technology? How do you use creative agencies to maximise the number of interaction points you have with customers with the right creative?” she says.

“We've had this challenge of being able to send the right message at the right time through the right channel for a number of years. And up until this point, we weren't able to present scalable creative at bottom of the funnel. That's what I'm excited about – scalable, personalised creative at the bottom of the funnel. I want to be able to augment my service proposition, create good content that my customers would like to see – things like financial wellbeing, financial tips,” says Ristevski.

“But they have to be aligned to the broader brand message, the broader emotional priming, the what is it that my brand represents? And that's the challenge. It's as simple and as complicated as that.”

Customer privacy, customer consent, ethical considerations, ‘one way, same way’ … Get everything signed off, [then] you know you’re not going to stuff yourself.

Suzana Ristevski, CMO, NAB

More… and better?

The bank is now using its new tech to get the basics of personalisation right.

“If we know a customer has been somewhere in the app, or has started an application on an equipment finance loan, and they jump on our website, it would be silly for us to serve a credit card. You want to serve them information that that is relevant to them. And you want to serve creative that is beautiful, resonates and does the job,” says Ristevski.

“So that’s what I mean by personalised content. I'm not saying just throw out more messages for the sake of it. I'm saying I can do more scalable messages now because I am banking on the fact that the creative and the content is relevant and suitable for what our customers want to see.”

Risk incoming

But there are risks in putting AI at the heart of business processes.

“The ethical considerations are really interesting, because when you're relying on AI algorithms, they can inadvertently perpetuate biases – decisions are made based off decisions that were previously made,” says Ristevski. “We need to make sure as a marketing community that we're not perpetuating bad biases. We must make sure that there's always that human oversight.”

AI or otherwise, there are some pretty punchy risks inherent to data-driven marketing and personalisation.

“Getting customers’ permission to actually market to them, protect their data, de-identify [them] … it all goes pear shaped if we don't do those things. So we're super, super paranoid about all of that. Which means from a governance perspective, no one in the organisation is allowed to introduce new technology without going through a formal process,” says Ristevski. “Customer privacy, customer consent, ethical considerations, ‘one way, same way’ … Get everything signed off, [then] you know you’re not going to stuff yourself.”

Either way, she said while marketing is spearheading a lot of the early work, NAB’s approach to AI governance and implementation goes straight to the most senior management echelons.

“We are starting from the top here. This is an executive strategy.... There are different use cases, marketing just happens to be part of the organisation that is first. But our risk teams are looking at it, our finance teams are looking at it; we are taking it seriously enough to bring it up to exec level, work out what the strategy is at that level and then filter it down.”

The biggest problem with [AI large language models] right now is the hallucination – basically it makes things up. Our first version was wildly wrong. Even when researchers asked ‘when was Tabcorp founded?’ it gave us totally wrong answers.

Amy Shi-Nash, Chief Data & Analytics Officer, Tabcorp

AI ‘hallucinations’

Tabcorp, while already developing machine-learning algorithms to help predict problem gambling patterns, is also acutely aware of the risks inherent in deploying artificial intelligence. It's building out a centralised 'Next Labs' to drive experimentation and work out wider use cases.

The wagering firm can’t use ChatGPT due to “privacy and security issues”. Instead, it has built its own large language models and is now into its third iteration, says Chief Data & Analytics Officer Amy Shi-Nash – who spent a significant chunk of her career leading data analytics at the likes of NAB, Commbank and HSBC. There are still some major wrinkles to iron out.

“The biggest problem with it right now is the hallucination – basically it makes things up,” says Shi-Nash. “Our first version was wildly wrong. Even when researchers asked ‘when was Tabcorp founded?’ it gave us totally wrong answers.”

Tabcorp changed the models and now the in-house generative AI is “starting to improve and get the basic facts right,” says Shi-Nash. But there’s a way to go, given the nature of Tabcorp’s business. “If you’re talking about a sports context, for instance, it's important to get it right.”

She says curating the data feeds to bespoke large language models – and then interpreting what it spits out – will be critical for businesses that cannot risk plugging into the likes of ChatGPT. “Out of the box AI is not there yet.”

Shi-Nash thinks improving the understanding of how generative AI works will be critical. She says “literacy” of its many quirks and shortcomings “is going to be a make or break factor in AI adoption”.

Who’s responsible for AI?

Tabcorp is building its “own in-house view” of the key AI use cases, risks, ethics, and where the lines of responsibility lie. For now, Shi–Nash said it’s pretty much all hands on deck.

“Risk and legal [departments] have a really key role, but also data and customer [functions] are a part of building that point of view. So it’s a joint ownership at the moment for strategy … or talking to regulators… or on the privacy side … you will have different parts of the organisation leading [on those different aspects].”

Her key takeout for marketers and brands grappling with generative AI and large language models? “Invest in the expertise, improve literacy, experiment – and learn fast.”

I fear not for the creative community in this evolution. I've shared an idea recently with Nick [Law, Accenture Song’s top creative], that I hold up as good as any we've ever created. And at its foundation is generative AI.

Mark Green, Accenture Song ANZ President & Group CEO, The Monkeys

Don’t fear the reaper

Fresh back from Cannes with a bagful of awards, Mark Green, Accenture Song ANZ President and Group CEO of The Monkeys, said the advertising festival was awash with AI chat.  “The amount of conversation around AI and generative AI was immense,” per Green.

There was as much fear and loathing as AI positivity among the great and good of creativity. But Green’s not worried.

“I’m always optimistic about the potential of technology and creativity … I don't want to be that cheerleader, but I do think there is, with caution and skill, a big opportunity here … the power of generative AI is significant. It’s going to be a game changer.”

He thinks businesses are already leveraging generative AI to good effect.

“Clients are using it in practical and sensible ways – experimenting with virtual shopping assistants, assembling recipes on behalf of food manufacturers,” says Green, nodding to NAB’s use of chatbots to service its customers.

“I think we're at that stage where almost every client is formulating a position or investing in it. So it's very real. It's not something that's going to happen in the near future. It's happening right now.”

He says the briefs are coming in from all angles – making AI something of a cross-functional unifier.

“There are briefs coming into our implied intelligence team, they are coming into our contact centre team, they are coming into our creative and content team … they are coming into our commerce team … so its application is quite wide,” says Green. “When you’re talking about generative AI, you find yourself surrounded by a whole bunch of different people.”

“So I fear not for the creative community in this evolution,” prophesises Green. “I've shared an idea recently with Nick [Law, Accenture Song’s top creative], that I hold up as good as any we've ever created. And at its foundation is generative AI. Some of the most interesting work we are involved with right now has some form of implied intelligence at its core.”

If the [AI-driven] middle is infinite and mediocre, then the best stuff, done with artfulness and intention, is going to be even more valuable.

Nick Law, Creative Chairperson, Accenture Song

Crap tsunami incoming?

Nick Law, Brooklyn-based Creative Chair at Accenture Song, agrees that fear that the robots will take creatives’ jobs is misplaced.

“There’s this is moral panic around [AI], because people are making the ridiculous assumption that you're not going to need creative thinking, that we’re going to recycle everything from a large language model,” says Law.

He thinks that will just lead to an explosion of average, with the good stuff becoming more prized as a result.

“The problem right now is that we're so astonished by the parlour trick of being able to create things from a prompt, that we're not really looking at the output with any sort of discernment. It's a little bit like when film first came along, it was enough to see a moving image,” per Law. He compares the current hoo-ha to the advent of Photoshop.

“If you look at the stuff coming out of [AI image generator] Midjourney, it looks like a 13-year-old’s idea of sci-fi. That’s because it’s largely done by an engineering culture. We haven't got to a point where the creative community is using it and mastering it enough for us to actually look at the output from these models clearly, we're still just astonished by [the capability to produce images from prompts]. “

However, Law thinks that will quickly change.

“I think we're going to look back next year from Cannes and see a whole body of [AI-powered] work.”

For brands, the challenge will be sorting the wheat from the chaff – which Law thinks requires more upfront discipline at risk of spewing out a “frantic spiral of abundance” and a long tail of crap. Law says one of the key lessons from working with Apple – he spent two and half years with the firm as VP of Marcom Integration – was “understanding the value of saying no to things”. He thinks the same applies to generative AI-driven creative output.

“I think that that's going to become a much bigger part of the process of creation. Because there is a scenario where the long tail of mediocre content is going to be so long, that the average audience for all the content on the internet is going to be one – which will be the author of the content,” suggests Law.

“So if the middle is infinite and mediocre, then the best stuff, done with artfulness and intention, is going to be even more valuable.”

You can do as much personalised messaging as you want – if the quality is not good, it doesn't matter ... Where I think we need to temper our enthusiasm in mass personalisation is when we get further up the funnel ... if we atomise our audience ... then we don't have a brand anymore.

Nick Law, Creative Chairperson, Accenture Song

The dangers of personalisation at scale

Law thinks similar rigour must apply to attempts to deliver personalisation at scale – or brands risk “atomising” what they actually stand for in the first place. He suggests advertising’s last major wave of disruption – programmatic performance marketing – provides a cautionary tale.

“We divided ourselves into sort of a tribe of a brand thinkers at the top that were very artful and making beautiful things that no one was seeing, because they lost touch with technology and modern media. And then there was this emerging tribe of people that really understood modern media and technology, but were making really mediocre things that everyone's doing,” says Law.

“You can do as much personalised messaging as you want – if the quality is not good, it doesn't matter. Where I believe that personalisation has the greatest power is when there is utility attached, where problems are very specific … And this is why AI is so useful in customer service,” says Law.

“Where I think we need to temper our enthusiasm in mass personalisation is when we get further up the funnel, because a brand in the end is a common understanding of the value of a company. And if we atomise our audience so much that we don't have that common understanding, then we don't have a brand anymore,” he adds.

“That's why I still think there's power in upper funnel brand building complemented with this stuff, which is more closer to the ground, more useful, more personalised. But I worry that if everything becomes about mass personalisation, then all of a sudden the brand disappears into mist.”

Law thinks that the ad industry can make “beautiful mass personalisation”, but must shift toward a more modular mindset, “to make the pieces beautiful, make sure the pieces fit together with a sense of unity”.

Which ultimately means “we need to treat these mediums the sort of care and creative excellence as we've applied to the top of the funnel”, per Law. And he’s not the only one saying that.

Mark Green thinks the time to better link upper funnel brand comms with the business end of performance is long overdue. Otherwise the vanilla, average experience associated with personalisation at scale will remain unchanged.

“We've got to be thinking about how humans are interacting with a company. Whether it is the mass personalised message from a bank or a big brand message, they all need a strategy, they all need to be deployed with care,” says Green.

“It all got a little bit lazy in marketing there for a moment, where we could just insert our message and no one had a choice not to kind of hear us or not.”

Experience is only applicable to customers. We have a job to do to be able to make sure that we maintain awareness and consideration and prospects. I'm certainly a very big fan of augmenting customer experiences. But our job is broader than that. We’ve got to bring people in.

Suzana Ristevski, CMO, NAB

Big brand ads still required

Despite building out a $45m tech stack to deliver non-crap personalisation at scale, NAB’s Suzana Ristevski isn’t backing away from traditional mass comms as a growth engine. She says any notion that customer experience can usurp brand building is “rubbish”.

“Experience is only applicable to customers. We have a job to do to be able to make sure that we maintain awareness and consideration and prospects,” she says. “I'm certainly a very big fan of augmenting customer experiences. But our job is broader than that. We’ve got to bring people in.”

Mark Green agrees. His key takeout as generative AI starts to disrupt marketing and advertising?

“Use the technology, don’t forget about the storytelling.”

What do you think?

Search Mi3 Articles