AI’s ability to see ‘mirages’ shows how alien machine brains really are

Intern, expert, or dog?

Ok, now, if you own a dog, as I do, there will be moments you’ll recognize that we fundamentally don’t understand how dogs perceive the world.

This week, while walking my dog, I spied a strikingly beautiful cat with an unusual coat. It looked like an orange tabby mixed with gray tabby, with a good deal of white fur thrown in the mix too. I noticed the cat right away, but it was moving across a yard that was elevated from sidewalk level, so my dog couldn’t see it. She could definitely smell it, however. She put her nose in the air and tugged at her leash, pulling her way up the steps that led to the yard.

By the time she got to the top step, the cat had mostly hidden itself behind a nearby flower pot. It stood behind the pot motionless, but with its white head popping above the pot’s edge. It stared intently at my dog and me. I could see the cat quite clearly. But, despite being just 15 feet away, my dog could not. She sniffed the air intently and pivoted first left and then right, but she could not see the cat, even when seemingly looking directly at it.

Eventually, I persuaded my dog to give up her hunt for the unseen, but well-smelled, cat, and continue our walk. But I couldn’t stop thinking about our differences in perception—and how this applies to AI. People often offer executives advice for how they should think about using AI by making analogies to our relationships with various categories of people. Treat AI agents like talented interns, was a popular one a few years ago, in the months following ChatGPT’s debut. A graduate student who is occasionally off their meds, was a colorful variant that Emad Mostaque, the cofounder and former CEO of Stability AI, liked to use. You should treat AI like PhD.-level researchers, was an analogy in vogue last year. (OpenAI CEO Sam Altman was among those talking about this idea.) More recently, people have started saying it is better to regard AI models like wise and experienced, but occasionally still fallible, colleagues. Certainly their performance on certain tough benchmarks of professional tasks, such as OpenAI’s GDPval, would lead one to endorse that idea. Middle managers is another analogy that comes up often.

But the more we learn about the large language models that underpin today’s AI agents, the more clear it becomes how inadequate all these analogies are. LLMs are nothing like people at all. They are far more like other species, like your dog. We can no more understand what and how these LLMs perceive and reach their outputs than we can truly understand the thoughts of our pets.

Actually, it’s worse than this, because unlike with our pets, you can ask an LLM to explain to you what it’s thinking and it will tell you. That sounds like a great thing, much better than the situation with our non-verbal dogs, cats, and turtles. The problem: Researchers have begun probing the activations of the artificial neurons in AI’s digital brains, and these experiments indicate that what an AI model tells you it is thinking—the model’s so-called “reasoning traces”—may or may not actually reflect what it is, in fact, thinking.

So interacting with an LLM is probably the closest thing we’ve had so far to interacting with an alien, one that has some capabilities that far exceed our own, but also has glaring weaknesses, and which can, at times, be just like us—deceptive, dishonest, or dissembling.

Multimodal models see ‘mirages’

This past week has brought yet more evidence of how weird these models are. A paper from researchers at Stanford University showed that multimodal AI models, those that can accept inputs in both text and images (and sometimes audio files too), suffer from a phenomenon they dubbed “mirage reasoning.”

The models will purport to analyze images a user has never actually uploaded to them. When prompted about medical images, but not actually supplied any images, the models will nonetheless offer diagnoses. Weirder still, these assessments are often correct. When the researchers tested the models on benchmarking tests for multimodal AI, the models obtained what the scientists said were “strikingly high scores”—about 70% to 80% of the scores they obtained when they did have access to images. Worryingly, the researchers found the models had a tendency to find evidence of pathologies in the phantom images, showing that the models may have a bias towards diagnosing disease that could lead to dangerous and expensive misdiagnoses if used in real-world medical settings.

The models’ sight is weak; their text pattern finding, unparalleled

The researchers have no clear understanding of exactly why the language models engage in mirage reasoning, or why they can score so highly on the benchmarks even when the images are not provided. But one experiment they conducted does suggest a possible explanation. The researchers fine-tuned a version of the open source AI model from Alibaba, Qwen-2.5, on a public training set for a popular benchmark that is designed to test how well AI models can answer questions about chest X-rays. But they trained it on this set with the accompanying images removed. They picked Qwen-2.5 in part because, at just 3 billion parameters, it is a relatively small model and therefore easy to fine-tune. But more importantly Qwen-2.5 was released a year before the chest X-ray benchmark they were using debuted, which the scientists hoped would minimize the chance that the set of questions actually used for the test itself would have ended up in Qwen-2.5’s initial pre-training data. (This kind of “data leakage” is a real problem for validity of AI benchmarks and a reason they need to be continually updated; otherwise models just memorize the answers as part of their pre-training.)

Nonetheless, this fine-tuned version of Qwen-2.5 outperformed every frontier AI model tested on the normal, image-included version of the X-ray challenge. It also beat the scores of human radiologists by 10%. Again, even though it did not have access to any of the images! The scientist found the model, despite never seeing any images, offered “reasoning traces comparable to, and in some cases indistinguishable from, those of the ground-truth or those generated by frontier multi-modal AI models.”

This implies, the scientists said, that there are hidden patterns in the questions themselves, perhaps in their phrasing, or in the structure of how those questions appear in the benchmark test, that are too subtle for any human to detect, but that nonetheless are sufficient to allow the model to guess the answer. This, combined with the researchers’ other findings, seems to suggest that multimodal models barely use the visual inputs they are given at all and instead lean heavily on linguistic patterns even when being asked to analyze images. It also suggests, alarmingly, that most of the multimodal benchmarks may not provide a good measure of how these models will perform in real-world clinical settings. 

Again, this is totally bizarre and alien to the way humans work. This is like my dog, able to smell the cat, but not see it—while I relied on my sense of sight, but could smell nothing. Our tendency to wrongly anthropomorphize AI models may lead us to misdesign the systems we use to run and govern AI agents, with potentially bad consequences. It also speaks to the way AI systems continue to improve in capability but lag in reliability that I wrote about last week. We need to engineer our AI workflows for alien minds, not our own.

With that, here’s more AI news.

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

Before we get to the news, if you haven’t yet read my colleague Sharon Goldman’s magisterial feature story on how construction of Meta’s massive Hyperion data center is upending the lives of people who live in rural Richland Parish, Louisiana, drop whatever it is you are doing right now, and go and read it. Here’s the link. It’s a deeply reported and deeply nuanced portrait of what happens when a community suddenly finds itself living in ground zero for the biggest, most expensive infrastructure build-out in American history. 

FORTUNE ON AI

Nvidia’s Jensen Huang says ‘We’ve achieved AGI.’ But no one can agree on what that means. Why the most important term in tech remains hotly debated.—by Jeremy Kahn

Sycophantic AI tells users they’re right 49% more than humans do, and a Stanford study claims it’s making them worse people—by Marco Quiroz-Guiterrez

Commentary: I’m a CEO who oversees $9.5 trillion in spend data. AI’s winners are already decided—by Leagh Turner

Dell’s CFO is using AI agents to run his finance team—and has helped the AI business go from $0 to $25 billion—by Sheryl Estrada

AI IN THE NEWS

Anthropic won a federal injunction to prevent the Pentagon’s "supply chain risk designation"from coming into force, but legal uncertainty remains. A U.S. federal judge in California granted Anthropic a preliminary injunction blocking the Trump administration’s ban on government use of its AI, pausing the policy while the case proceeds. Judge Rita F. Lin questioned the government’s rationale for labeling the company a “supply-chain risk,” a designation that had never previously been applied to a U.S. company, suggesting the move was not driven by national security concerns but may constitute illegal retaliation against the company. You can read more about the decision here from Bloomberg News. But Anthropic isn’t out of the woods yet. The government said it would appeal Judge Lin’s decision. What’s more, the government rested its supply chain risk designation on two different statutes, one of which can only be overturned by a three-judge appellate panel that sits in Washington, D.C. Anthropic has filed suit before this court too, but the appellate panel has yet to set a date for hearing arguments in the case, and, as legal experts tell Politico, Anthropic faces potentially far-less-sympathetic judges in the D.C. case than it did in  California.

OpenAI ditches video generation app Sora, and loses $1 billion from Disney. OpenAI made the abrupt decision to shutter its Sora video generation app in order to dedicate more computing and talent to its core ChatGPT and Codex coding agent efforts. Sora used huge amounts of computing power, while the app had seen user numbers plateau and was deeply unprofitable, the Wall Street Journal reported. That drag on profits and scarce computing resources was something OpenAI could not afford as it prepares for a landmark IPO, perhaps as soon as later this year, and as it faces stiff enterprise competition from Anthropic. But the newspaper also reported that OpenAI gave the Walt Disney Company almost zero warning of its decision, even though Disney had planned a $1 billion investment into OpenAI and a licensing deal that would have allowed Sora users to generate images using Disney characters and IP. The Sora shutdown leaves OpenAI retreating from its ambitions in AI-driven entertainment, possibly opening up space to other AI players. It also effectively ends a budding relationship with Disney that might have helped OpenAI win over more investors.

Mistral AI raises $830 million to fund Nvidia-powered data centers in Europe. The Paris-based AI startup raised the money through debt financing, the Financial Times reported. Mistral, often seen as a “European champion” among frontier AI labs, has seen demand for its models grow as more countries in Europe and beyond look for “sovereign” AI alternatives to U.S. tech giants. The funding supports a broader multibillion-euro infrastructure push, with facilities planned in France and Sweden to meet rising demand from governments and enterprises seeking more control over their AI systems.

California governor issues AI executive order. Governor Gavin Newsom, considered a top contender for the Democratic presidential nomination in 2028, issued a first-of-its-kind executive order requiring AI companies seeking California state contracts to disclose detailed safety, privacy, and bias-mitigation practices, including protections against exploitation and misuse. The order also asserts the state’s independence from federal determinations—such as the Pentagon’s designation of Anthropic as a supply-chain risk—allowing California to make its own decisions about which AI vendors it works with. It further directs state agencies to begin watermarking AI-generated content to combat misinformation and improve transparency for the public. The order potentially puts California further in the cross-hairs of the Trump administration, which has vowed to use federal power to punish states that enact their own AI laws. Read more from the New York Times here.

FTC settles with dating site operator Match Group over sharing of photos with AI company. Match Group agreed to settle a lawsuit by the Federal Trade Commission over allegations that its OkCupid platform shared users’ personal data—including photos and location information—with controversial AI firm Clarifai without proper consent. The FTC said this violated OkCupid’s own privacy promises, which stated user data would not be shared with unauthorized third parties. Under the settlement, Match and OkCupid must stop misrepresenting their data practices and privacy controls, but no financial penalties were imposed. Clarifai is still allowed to use the data and the facial-recognition models trained on that data. Clarifai’s tech has been used by law enforcement agencies across the U.S. to help identify potential suspects, despite concerns from civil rights advocates that the technology has, in some cases, led to false arrests and that it generally violates civil liberties and privacy norms. The FTC’s move to settle with Match also follows an executive order on AI by President Trump, issued last summer, that mandated the agency review all investigations begun under the Biden administration to ensure “they do not advance theories of liability that unduly burden AI innovation.” Read more from Reuters here.

EYE ON AI RESEARCH

Could AI help moderate political polarization? One of the impacts of chatbots that I’ve been most concerned about is their potential to exacerbate filter bubbles. The models tend to be sycophantic, telling people what they want to hear. Will they actually challenge people’s views, or reinforce them? My fear has been the latter. It would be like social media on steroids. But analysis out this week from data journalist John Burn-Murdoch at the Financial Times argues that unlike social media—which tends to amplify extreme and polarizing views—AI chatbots may have a “technocratising” effect, nudging users toward more moderate, expert-aligned positions.

Drawing on a large dataset of political attitudes, Murdoch tested how leading chatbots (including Grok, GPT, Gemini, and DeepSeek) influence discussions across 61 policy and social issues. Across thousands of simulated conversations, all models consistently shifted users away from ideological extremes: Grok leaned slightly center-right, while ChatGPT, Gemini, and DeepSeek leaned center-left, but all reduced fringe views on both sides.

The study also found that chatbots rarely endorse conspiracy theories—such as election denial or vaccine misinformation—contrasting sharply with social media, where such beliefs are over-represented. Importantly, these moderating effects persisted even when bots were given information about users’ political leanings, suggesting the influence is not just a result of tailoring responses to users.

The findings suggest that, if used correctly, AI chatbots could counteract polarization. That would be a rare bit of good news from our embrace of AI.

AI CALENDAR

April 6-9: HumanX 2026, San Francisco. 

June 8-10: Fortune Brainstorm Tech, Aspen, Colo. Apply to attend here.

June 17-20: VivaTech, Paris.

July 6-11: International Conference on Machine Learning (ICML), Seoul, South Korea.

July 7-10: AI for Good Summit, Geneva, Switzerland.

BRAIN FOOD

Will countries that lag in adoption face stagflation? That’s the prospect being raised by some recent analysis from Luis Garicano, an economist at the London School of Economics. Garicano argues that AI will lift productivity in the economies that adopt it most aggressively, particularly the U.S. and China. As the economy booms, these countries will need to raise interest rates to prevent inflation. But Garicano argues that, because of the way capital markets tend work, and especially given that the U.S. is still, for the moment, the leading global reserve currency, these high rates will nonetheless be “imported” into countries that don’t adopt AI as rapidly. These regions, he argues, will face the prospect of higher interest rates, but without the compensating balm of high growth. They will in essence be facing possible stagflation. He says this prospect is especially alarming for places like Europe and the U.K., which are already suffering from low productivity growth and where AI adoption is lagging the U.S. and China. You can read his analysis, which he calls “R without G” on his blog here.