What you need to know:

• CEO Dario Amodei revealed that an internal AI agent, "Claudius", nailed inventory, pricing and procurement – including a bizarre tungsten cube request – but caved when users begged for discounts, exposing how easily current AI can be manipulated.
• The Anthropic chief warned that agents lacking psychological resilience are vulnerable to social engineering. Enterprises should beware of mistaking procedural competence for human-level understanding.
• His comments came just days after securing a massive funding round and agreeing to compensate content creators whose work trained Anthropic’s models.
• Perplexity’s Srinivas pushed back on his investor's appetite for  ‘hallucination as a feature’: Facing pressure to prioritise virality over accuracy, the CEO says he stood firm, committing to verified answers and citations over flashy but false responses.
• The moral of both accounts: AI is dazzlingly capable but dangerously naïve. Without containment and responsibility, it risks creating liabilities.
• Srinivas forecasts a shift from chatty responses to agents quietly doing executive grunt work. But he also hinted that ethical safeguards may depend more on investor incentives than AI design.

Two of the LLM world’s emerging leaders, Anthropic’s Dario Amodei and Perplexity’s Aravind Srinivas, highlighted the limitations of their machines – and in Perplexity’s case, their owners' grasp of Responsible AI during their keynotes at last week’s Inbound event in San Francisco.

Amodei described the results of 'Vend', an internal experiment to build an agent that could run a shop but melted when a customer pleaded for a discount. Srinivas recalled pressure from his investors to let his product trade accuracy for viral thrills – he refused.

Together, the stories they told the attendees at the annual HubSpot customer conference offer cautionary clarity about how much we should trust today’s AI, and why it’s important to keep a human hand on the tiller.

Amodei, fresh from raising $13 billion two days earlier and then handing over more than a billion the next day to the publishers and authors whose IP Anthropic lifted to train its models, leaned into Claudius for his talk. It was an agent designed to run an internal vending machine business at Anthropic, and it proved preternaturally competent, at least on one side of the equation. It trawled catalogues, re-ordered stock, and repriced with a tidy grasp of supply and demand.

It even indulged eccentric demand: “One person wanted, for some reason, a giant tungsten cube, a cube of solid tungsten, and Claudius managed to procure it,” Amodei said.

Yet when it came to the very human business of buyers and sellers haggling over price, the agent proved a pushover at the counter.

“Claudius was brilliant at finding and ordering things that people asked for… but very ridiculous when people begged it for a discount.”

His point: “This was a very limited form of agency,” and enterprises that mistake it for maturity will be disappointed. The harder frontier lies in building models that can resist manipulation as ably as they manage stock. Negotiation and persuasion, after all, require a grasp of human psychology. Without it, machines remain dangerously easy to trick. Future systems, he suggested, must fold in safeguards against social engineering and sharper models of human intent.

For now, perhaps, the answer may be to treat AI less as a tireless colleague than as a precocious but erratic intern: dazzling one moment, credulous the next.

Irresponsible AI

Perplexity’s Srinivas, meanwhile, opened the kimono a little on how the investors funding the boom in agentic and generative revolution really behave when Responsible AI slips out of the corporate PowerPoint presentation and slides inconveniently into the world of real things.

"We were actually discouraged a lot by investors right before launching Perplexity. ChatGPT had just gone live (30 November 2022), and a week later, on 7 December, we were ready to launch. I gave early access to our consumer-focused investors," he said.

"ChatGPT was going viral because people loved seeing it make mistakes. People were sharing screenshots of AI failures, it was almost entertainment. My investors looked at that and asked: Why aren’t we doing the same thing?”

Srinivas told attendees that the investors complained that Perplexity was making the answers more boring, general, and very academic. "You need to build up a massive audience. Other products are going viral, because people love hallucinations. In your product, hallucination is a bug, but what people want is hallucination as a feature, so you've got to [do likewise], you shouldn't be launching this," they told him.

"Now imagine we listened to the advisors and launched a bot that was more like a psychopath?"

Srinivas said the principle guiding his decision was clear: “Only an accurate answer serves as a foundation for the next set of questions.”

That, in turn, shapes how people should use AI at all. Good leaders, in his telling, do not arrive with pronouncements; they arrive with better questions prompted by reliable answers.

Set side by side, these CEO confessions sketch an emerging division of labour. Give agentic systems bounded, procedural work and watch them hum; ask them to withstand social pressure and you risk whimsy turned costly. Equally, celebrate answer-machines that go viral for the wrong reasons, and you corrode the very trust that makes them useful.

The thread that ties both talks together is discipline: containment and guardrails for agents; accuracy and citations for generative search engines

From answers to actions

Srinivas, meanwhile, looked beyond conversational AI into the emerging agentic era. “The last three years of AI have been all about giving you great conversational answers… the remaining part of this decade is going to be about moving from answers to actions.” The difference is more than semantic.

The Perplexity chief describes a world where agents shoulder the chores executives quietly loathe: dredging Slack threads, formatting reports, digesting filings, scheduling the endless call.  It’s a world where software companions anticipate needs and act.

Whether they are built to act responsibly may sometimes depend less on ethics, and more on the blunt priorities of investors.