Two companies that run AI models on behalf of other businesses disclosed breaches this week, and the damage landed on their customers' customers. Fireworks AI, which says it serves open models to tens of thousands of companies, said attackers who first broke into its cloud provider took billing information and some datasets, as TechCrunch reported(opens in a new tab). Notion then confirmed that Notion AI chat transcripts kept for its Insights feature were taken, in fewer than 1% of workspaces, according to Notion's advisory(opens in a new tab). Uber told drivers that their license details were taken by an attacker using internal credentials belonging to the same provider. State filings count at least 7,870 affected drivers in South Carolina, Massachusetts and Rhode Island alone, IBTimes reported(opens in a new tab), and Uber has not given a national total.
Separately, NEAR AI said an attacker used a flaw in SGLang, free software that runs AI models on servers, to run code on one of its inference servers on November 8, per The Hacker News(opens in a new tab). An inference server is the machine that actually computes the answer to your prompt. NEAR AI sells a confidential mode that processes requests inside a hardware-sealed area hidden even from NEAR AI itself. The attacker could still have seen those requests, because the flaw sat in the software running inside the sealed area.
The ROAI point is about where your prompts sit after they have been answered. A Notion user who typed into Notion AI never chose Fireworks. Notion kept chats so its Insights feature could analyze them, and that stored copy is what leaked. Uber's drivers never chose Fireworks either. Every AI feature in a tool you already pay for may pass your prompts to a company you never signed with, and some keep prompts and answers afterwards for analytics, testing or training. Each stored copy is one more place to breach, and the bill arrives as notification letters, credit monitoring, rotated credentials and lost trust.
The cheapest control is fewer copies. Switching off an optional feature that stores chats removes one. And a confidential or private label tells you who is not supposed to see your data. It does not tell you whether the software that reads it is patched.
Action items
A breach at Fireworks AI, a company that runs AI models for other businesses, reached Notion AI chat transcripts and Uber drivers' license details, and a flaw in open inference software could have exposed requests at NEAR AI that were sold as confidential. The risk is not only which AI tool you use, but who keeps a copy of what you typed into it.
For the CISO, and for anyone who uses AI at work, this week. Pick the three tools where your people use AI features most and ask each vendor two questions in writing: which outside companies process our prompts, and does anyone, you or them, keep prompts and answers after the reply, for how long and for what purpose. Switch off optional features that store chats for analytics or review where you can. And if you have ever pasted a password or API key into an AI chat, change it now; Notion is telling affected customers to do exactly that.
Researched and drafted by an automated workflow, then reviewed and edited by a human editor before publication. Every source is linked. See how we use AI here.
Also worth knowing
- NotionNotice about the Fireworks AI incident that may impact some Notion workspaces(opens in a new tab)
Notion says Fireworks exposed Notion AI chat transcripts kept for its Insights feature in under 1% of workspaces. Insights is off for now. If you pasted passwords or keys into Notion AI, Notion says to rotate them.
- IBTimesUber Drivers' License Details Stolen in Major Data Breach. Thousands Affected Across US.(opens in a new tab)
At least 7,870 drivers in three states had license details taken in a breach that came through Fireworks AI's credentials. Uber offers free credit monitoring through Cyberscout, with 90 days to enroll. A license number is harder to replace than a password.
- The Hacker NewsSGLang RCE Exploit Breaches NEAR AI, Steals Data From 18 Agents and About 100 Users(opens in a new tab)
A flaw in SGLang, open software that teams use to run their own models, let an attacker run code on a NEAR AI server and reach agent chat logs and two API keys. If your team runs models on SGLang, find out which version and who can reach it.
- GoogleIntroducing Gemini 4 Hydrogen for Agentic Workflows(opens in a new tab)
Google's new fast model costs $1.50 per million input tokens and $7.50 per million output, flat across a 1 million token context. Free app users get it only when automatic selection picks it. Google says it edges its Argon model on one index, 51 to 50, by its own count.
- California Management ReviewYour AI Productivity Gains May Be Hiding a Capability Gap(opens in a new tab)
Christoph Trattner argues teams should measure what staff can still do when the AI is down or wrong, not just output with it. He suggests testing three to five high-stakes workflows without AI over 90 days.