When Salesforce recently rolled out an AI agent on its website, the agent started to hallucinate and wasn’t giving consistent results.
Salesforce ended up temporarily turning it off, Shibani Ahuja, senior vice president of enterprise IT strategy, said during a roundtable discussion at Fortune’s Brainstorm Tech conference in Park City, Utah.
But the agent, it turned out, wasn’t the problem. “What we had noticed was there was an underlying problem with our data,” Ahuja said. When her team investigated what had happened, they found that Salesforce had published contradictory “knowledge articles” on its website.
“It wasn’t actually the agent. It was the agent that helped us identify a problem that always existed,” Ahuja said. “We turned it into an auditor agent that actually checked our content across our public site for anomalies. Once we’d cleaned up our underlying data, we pointed it back out, and it’s been functional.”
New AI products will only be as good as the underlying data, according to Ahuja and other speakers who took part in the discussion. Ashok Srivastava, senior vice president and Chief AI Officer at Intuit, said he wasn’t surprised about the results of a recent MIT study that found that 95% of AI pilots at large corporations had failed, because of the archaic systems at large companies.
“The fact is that the foundation of AI—which is data—people don’t invest in it,” Srivastava said. “So you’ve got 1990s data sitting in a super-expensive, unnamed database over here, you’ve got AI here, you’ve got the CEO telling you to do something, and it’s just not going to work.”
Sean Bruich, senior vice president of artificial intelligence and data at Amgen, added that it’s also difficult for larger corporations to move from a pilot to enterprise-wide adoption.
“Pilots in large companies never deliver ROI,” he said. “They might deliver learnings, they might deliver proof points, they might deliver inspiration. But the path to scale—that is where you get the return on investment in any large technology program.”
In order for companies to see a return on investment from new AI tools, they will have to sort through both the data and the scaling issue.