Security experts gather in D.C. to debate how to secure AI systems as Mythos raises the stakes

Welcome to Eye on AI, with AI reporter Sharon Goldman. In this edition: Top Republican pushes party to shun $300 million AI lobby…AI model scams are scary good….Anthropic’s new AI model sets off global alarms.

As Anthropic Mythos drove a fresh wave of headlines this week—highlighting both its advanced capabilities and how easily such systems could be misused—I made my way to a conference room just outside Washington, D.C. There, a cross-sector group of AI security practitioners, standards-setters, and policy experts had gathered to figure out what securing AI should actually look like.

Outside the industry, their acronyms—SANS, NIST, OWASP, CoSAI, CIS, CSA, BIML—may not mean much. Inside security, they help set the rules organizations around the world follow. But right now, those rules are struggling to keep up.

I had been invited to sit in on the discussion as organizations race to plug AI into everything—handing over sensitive data and critical workflows—even as those same systems are becoming more attractive targets for adversaries.

Leading the session was Rob van der Veer, chief AI officer at software platform Software Improvement Group and a founder of the AI Exchange at security community OWASP. Systems like Mythos, he said, are accelerating how quickly vulnerabilities can be discovered—and shifting the balance toward attackers.

“They show that weaknesses in AI systems can now be found faster and at scale—often before developers are aware of them,” he said. “This shifts the balance toward attackers and reduces the margin for error.” So far, concerns about Mythos have mostly focused on how good it and similar models are at finding so-called “zero-day” vulnerabilities in traditional software, but they can also discover vulnerabilities in the AI models and systems that enterprises are increasingly deploying across their organizations.

The problem is that most organizations aren’t ready to deal with most of the AI security concerns that are already clear and the emerging ones coming down the pike. There’s a growing need for practical guidance—how to identify AI-specific threats, and what to do about them. But the field remains fragmented, with overlapping frameworks, competing recommendations, and little agreement on where to start.

How to secure AI systems is still unsettled

Even some of the basics are still unsettled. What does it mean to measure whether an AI system is secure? How should that differ across use cases, infrastructure, or third-party tools versus underlying models? Should guidance focus on capabilities, or outcomes?

Gary McGraw, cofounder of the Berryville Institute of Machine Learning, pointed to a core gap: Today’s benchmarks tend to measure how well AI systems can perform security tasks—not how secure the systems themselves are. Companies need to keep that distinction in mind when evaluating their tools and defenses.

McGraw warned as far back as 2019 that securing machine learning systems would be “one of the defining cybersecurity struggles of the next decade.” That moment has now arrived.

“These meetings are a way to remind ourselves of the fundamentals,” he said, “as we try to define what machine learning security actually is.”

Another significant concern is that no finite set of guardrails is universally robust against adversarial prompts, said Apostol Vassilev, a research team supervisor working on AI security at the National Institute of Standards and Technology (NIST), part of the U.S. Department of Commerce. “This means that the security of AI systems is not a static problem—one that can be solved once and done,” he said. Unlike many traditional software vulnerabilities that can be patched, AI security requires a more dynamic approach: continuously updating guardrails to address known exploits, conducting internal red teaming to uncover new adversarial prompts, patching defenses before attackers strike, and prioritizing resilience so enterprises can limit the impact of—and recover quickly from—inevitable exploits.

“Ultimately, the goal is to reach an equilibrium that makes it difficult and costly for attackers to find new exploits,” he added. “But that can only happen if businesses invest in adopting and maintaining this dynamic posture.”

Similar to transition to securing software

Still, many of the meeting’s attendees remain optimistic the industry will catch up. McGraw noted that security has been through transitions like this before, such as the software boom of the mid-90s. “We didn’t have to panic when software swamped the world,” he said. “I remember when banks realized, ‘Oh my God, we’re a software company.’”

At moments like this, the narrative communicated by companies like Anthropic and OpenAI can run ahead of the reality, he warned. “Security loves a good story with a flaming pile of broken stuff and the fire department coming to the rescue,” he said. “I am still optimistic that we’re making progress towards better security engineering all the time. We can take what we’ve learned and we can apply that to machine learning.”

And that’s why these kinds of meetings about industry coordination matters, said van der Veer. “Aligning standards and guidance across initiatives reduces fragmentation, improves clarity, and gives practitioners a coherent path forward,” he explained. “It enables organizations to move fast without losing control.”

With that, here’s more AI news.

Sharon Goldman
sharon.goldman@fortune.com
@sharongoldman

FORTUNE ON AI

The AI boom is single-handedly carrying the U.S. import market—and adding $200 billion to the trade deficit, Fed study finds – Tristan Bove

The European AI unicorn run by a baker’s son—he learnt the fundamentals of business watching his father make bread rolls – by Kamal Ahmed

Cursor’s 25-year-old CEO is a former Google intern who just inked a $60 billion deal with SpaceX – by Marco Quiroz-Gutierrez

Meta will start tracking employees’ screens and keystrokes to train AI tools – by Eva Roytburg

Investors continue to punish ServiceNow despite strong earnings and CEO McDermott’s forecast of blistering growth in AI product sales — by Jeremy Kahn

AI IN THE NEWS

White House memo focuses on protecting American AI from "distillation" attacks. In a new White House memo, Michael Kratsios, director of the office for science and technology policy, said that the U.S. has evidence that foreign entities, primarily in China, are running industrial-scale distillation campaigns to "steal American AI." The memo emphasizes maintaining U.S. leadership in AI, protecting critical infrastructure and supply chains from foreign adversaries, and expanding coordination across government and with allies—all while ensuring systems are “safe, secure, and trustworthy” and aligned with democratic values. It also highlights the growing role of private-sector AI development, calls for more talent and compute infrastructure, and warns that failing to act quickly could allow rivals to close the gap, framing AI not just as a technology shift but as a defining geopolitical competition.

AI chipmaker Cerebras files to go public after scrapping IPO plans last year. CNBC reported that Cerebras Systems has filed to go public on Nasdaq (ticker: CBRS), reporting a sharp turnaround to $87.9 million in net income on $510 million in 2025 revenue (up 76% year over year), while signaling a major strategic shift from selling chips to operating them as a cloud service—putting it in more direct competition with hyperscalers. The filing also highlights significant customer concentration, with UAE-linked institutions including Mohamed bin Zayed University of Artificial Intelligence accounting for a large share of revenue. At the center of its growth plans is a more than $20 billion compute deal with OpenAI, backed by a $1 billion loan and equity warrants—but one that depends heavily on Cerebras delivering massive infrastructure on time, with OpenAI retaining the option to walk away if it doesn’t.

Top Republican pushes party to shun $300 million AI lobby. The Financial Times reported on growing political backlash against pro-AI money in U.S. elections, as Josh Hawley urged Republicans to reject funding from well-funded AI-linked Super PACs backed by figures tied to OpenAI, Palantir Technologies, and investors like Andreessen Horowitz. With war chests reportedly reaching hundreds of millions, critics—including Bernie Sanders and Alexandria Ocasio-Cortez—warn that AI industry influence is chilling debate and discouraging regulation ahead of midterms. Hawley argued that unchecked AI poses risks ranging from harm to children to economic strain, while blaming bipartisan deference to Big Tech for stalled legislation. The backdrop is a broader tension in Washington as the White House pushes for national AI dominance and resisting state-level rules, while lawmakers from both parties call for stronger safeguards.

5 AI models tried to scam me. Some of them were scary good. This is an interesting piece from Wired reporter Will Knight, who describes firsthand how advanced AI models are becoming alarmingly effective at social engineering, recounting a highly personalized phishing attempt generated entirely by the open-source model DeepSeek-V3. Using a testing platform from Charlemagne Labs, Knight shows how multiple leading models—including Claude 3 Haiku, GPT-4o, and others—can convincingly impersonate attackers, adapt in real time, and scale scams, even if they sometimes fail or refuse outright. The article argues that AI’s conversational strengths—like flattery and contextual awareness—make it particularly dangerous in phishing and fraud, potentially automating large parts of the attack “kill chain,” even as parallel advances like Mythos raise both defensive hopes and new risks.

EYE ON AI NUMBERS

80%

That's how many U.S. adults who reported using Claude in the previous week live in households that earn $100,000 or more a year, according to Epoch AI research.

In comparison, only 37% of Meta AI users earn $100K or more per year. Nationally, the research found about 50% of U.S. adults fall in this income bracket.

Among Meta AI users, 32% live in households earning less than $50,000, compared to 7% of Claude users and 24% of U.S. adults. Other major providers cluster in a relatively narrow band, with 56% to 64% of users in $100,000+ households.