An inference provider's second breach in a month reached billing records, datasets and incomplete model weights — and leaked through to Notion's customers. The uncomfortable part isn't the intrusion. It's how long the scope took to stabilise, and how far down the chain the blast radius travelled.
Last Updated: November 11, 2026
Fireworks AI disclosed on 9 November that a third-party cloud provider hosting its R&D machines was breached on 23 October, and that attackers used it the next day to reach Fireworks development machines. The scope of its earlier October incident also widened from “secrets in evaluation jobs” to billing data, resource metadata, and incomplete model weights and datasets. Notion, a Fireworks customer, has confirmed AI chat transcripts for under 1% of its workspaces were in the exposed data. If you integrate AI, the actionable lesson is not “avoid Fireworks” — it is that zero-data-retention on the inference path says nothing about the eval, fine-tuning and analytics pipelines sitting next to it.
What actually happened
The first incident is a month old. On 6 October, Fireworks detected unauthorised use of internal credentials that let a third party read environment variables used in evaluation jobs across a limited number of customer accounts. At the time, the company said it had found no evidence of access to customer models, datasets, training data, inference traffic or personal information.
That assessment did not hold. In an update published 9 November, Fireworks said forensic work with Palo Alto Networks Unit 42 found impacts to account and billing information, and for a limited set of accounts, to secrets, datasets and incomplete models. It also disclosed a second, separate intrusion: On October 24, the unauthorized third party used it to reach Fireworks development machines and then run an unauthorized workload that led to additional customer data impact.
The cloud provider that was the entry point is not named. Fireworks says it detected the activity the same day and has seen no further successful unauthorised access on the customer-facing platform since 25 October.
Fireworks has more than 10,000 customers, according to The Record, and the company attributes the activity to a financially motivated group monetising stolen model access — LLMjacking. BleepingComputer reported the second breach on 10 November.
Fireworks detects misuse of internal credentials; containment begins the same day. Customer notifications start 7 October.
First bulletin published. Employee API keys rotated, service-account tokens rotated, actor-created keys revoked.
Third-party cloud provider hosting Fireworks R&D machines is compromised; the following day the actor pivots into Fireworks development machines and runs an unauthorised workload.
Notion is told a small dataset was accessed — initially assessed as synthetic test data. By 6 November the assessment changes to potential customer data; confirmed 8 November.
Fireworks publishes the expanded bulletin. Notion publishes its own advisory. Press coverage follows.
The part that matters: the blast radius went four parties deep
Follow the chain. A cloud provider is compromised. That provider hosts Fireworks' R&D machines. Fireworks serves inference and classification jobs for Notion. Notion serves millions of workspaces. The Notion advisory says fewer than 1% of workspaces were affected, that the exposed data included AI chat transcripts used by its Insights feature, and that for the vast majority of impacted workspaces those transcripts were unattributable — not labelled with author or workspace, and containing no tool call results.
Notion still tells affected customers to treat the data as exposed and rotate anything sensitive pasted into Notion AI. That is the right call, and it is the tell. “Unattributable” is a property of the metadata, not the content. A chat transcript that contains a customer name, a contract clause or a pasted API key identifies itself regardless of what label was stripped off the top.
Most vendor risk registers I see model one hop. Your contract is with Notion, or with whichever SaaS tool your team adopted; Notion's contract is with Fireworks; Fireworks' contract is with a cloud provider nobody has named publicly. The breach notification you receive is three translations away from the forensic evidence, which is exactly why the scope took a month to stabilise.
Zero data retention is a data-plane promise, not a company-wide one
Fireworks says it has not detected impact to Inference, Training SDK or Nexus, all of which are zero data retention by default. Read that carefully: it is a statement about the serving path. The data that moved was in an internal application — resource metadata, billing records, and stored datasets and partial weights. ZDR means your prompts aren't persisted. It does not mean your fine-tuning corpus, your eval fixtures, your batch inference uploads or your billing address live under the same guarantee.
This is the single most common gap I find when auditing an AI integration. Teams get a ZDR commitment, file it, and then ship a fine-tuning job that uploads a CSV of real customer records to the same vendor.
| Surface | Typical guarantee | What the Fireworks incident showed |
|---|---|---|
| Live inference (prompts/responses) | Zero data retention by default | No detected impact, per Fireworks |
| Evaluation jobs | Rarely covered explicitly | Environment variables, including API keys and tokens stored as secrets, were captured |
| Datasets & fine-tune artefacts | Stored at rest by design | A subset of incomplete model weights and datasets accessed |
| Account & billing records | Out of scope of AI-specific terms | Addresses, invoices, payment history, balances accessed |
| Your vendor's vendors | Usually undisclosed | Entry point was an unnamed third-party cloud provider |
Why an API key is now worth stealing on its own
The economics changed. Google's threat intelligence team documented in early November an ecosystem where initial-access actors hand off to operators running “LLM Mixers” — illicit gateways that proxy and resell stolen model access to underground subscribers. Unlike cryptojacking's slow burn, this produces spikes in hours.
As enterprises integrate higher-tier models and agentic workflows into production environments, cybercriminals will likely refine automated reconnaissance to target exposed API tokens and exploit vulnerabilities in model routing middleware.
— Google Threat Intelligence Group and Mandiant, The Rise of LLMJacking
Practically: an unexplained jump in token spend is a security alert, not a finops ticket. If your anomaly detection on model spend runs monthly against an invoice, your detection window is four weeks wide and the attacker only needs a day.
Deleting the stored secret and calling it rotated
When a provider revokes its own keys, teams assume the job is done. But the third-party credentials you stored inside that provider — a database URL, an S3 key, a webhook token for an eval job — are still live at their issuers. Fireworks spells this out in its first bulletin: removing a stored secret alone does not invalidate the underlying credential.
Auditing only your direct inference contracts
Most organisations touched by this incident never signed anything with Fireworks. They used a product that did. Your SaaS subprocessor list is the real inference inventory, and it is almost always out of date because AI features get added between annual reviews.
The audit to run this week
Inference-provider exposure review
List every deployment that calls a hosted inference endpoint, directly or through a SaaS feature, a gateway, or an agent framework's default router. Record which provider, which region, and whether traffic is ZDR.
For each provider, list what you have uploaded that persists: fine-tuning datasets, eval fixtures, batch inference files, cached embeddings. That list is your real breach exposure, not your prompt volume.
One key per workload, never per team. Short expiry. Spend caps and rate limits per key so a stolen credential has a ceiling measured in hundreds of dollars, not six figures.
Daily thresholds wired to the same channel as auth anomalies. Include model mix — a workload that suddenly calls a frontier model it never used is a strong LLMjacking indicator.
Name the second provider and the switch procedure. Notion disconnected within roughly twelve hours; that is only possible if the alternative was decided in advance.
We now put an inference-provider risk section into every integration architecture document we ship — provider, retention posture, persisted artefacts, key scope, spend ceiling, named failover. It takes an hour to write and it is the difference between a one-day response and a one-month one. If you want help running that review across an existing estate, that is what our integration work covers, and the approach shows up in our case studies.
The honest caveats
Fireworks handled the disclosure better than most: same-day containment on both incidents, named forensics partner, a public bulletin, and a willingness to publish that the first assessment was wrong. The widening scope is what a real investigation looks like, not evidence of concealment. The alternative — silence until certainty — would have been worse for customers.
But three things remain unsaid. The number of affected customers is not disclosed. The upstream cloud provider is not named, which means other companies hosting R&D on it cannot check their own exposure. And every impact statement is framed as “we have not detected” — a claim bounded by log coverage, in an incident whose remediation list explicitly includes enhancing logging and monitoring. The investigation is ongoing and may reveal more. Plan on that, rather than on the current scope being final.
Frequently Asked Questions
Were prompts and responses exposed in the Fireworks AI breach?
Fireworks says it has not detected impact to Inference, Training SDK or Nexus, which are zero data retention by default, and that prompts and outputs are not stored by default. However, Notion confirmed that AI chat transcripts processed for its Insights classification feature were in the exposed data. The distinction matters: live serving traffic was not retained, but conversational data sent to a separate classification pipeline was.
Should we stop using Fireworks AI?
Not automatically. A provider that discloses, hires external forensics and corrects its own scope publicly is not obviously riskier than one that has had the same intrusion and not found it. The better questions are whether you can switch providers quickly if you need to, whether your keys are scoped and capped, and whether anything you uploaded persists at the vendor. If those three answers are weak, the provider choice is not your biggest problem.
What is LLMjacking and why does it change key handling?
LLMjacking is the theft of model credentials to resell inference access through illicit gateways, leaving the legitimate owner with the bill. Google's threat intelligence team observed fraudulent charges exceeding $100,000 in a single day in individual incidents. Because monetisation is immediate, a stolen AI API key is valuable on its own, independent of any data behind it. That makes per-workload keys, short expiry and hard spend caps a security control rather than a cost-control nicety.
How do we find out if we are exposed through a SaaS vendor rather than directly?
Start with each vendor's published subprocessor list and security advisory page, then check whether any AI feature you enabled routes to a third-party inference provider. In this incident Notion contacted affected workspaces directly and said no action is needed if you were not contacted. Treat absence of notice as provisional while investigations are open, and ask vendors in writing which inference providers handle your data today.
What should go into an inference-provider risk section of an architecture document?
Six fields: the provider and region, the retention posture for each surface used, a list of artefacts that persist at the vendor, key scope and rotation cadence, a per-key spend ceiling with an alert threshold, and a named failover provider with the switch procedure. Review it when the provider changes, when a new persisted artefact is added, and after any vendor incident — not annually.
Research digest
AI Research Briefing
Honest insights on AI agents, Small Language Models, and local RAG. No hype. Only when we have something worth sending.
- No hype, just measurable outcomes
- Read by 2,400+ engineers
- Unsubscribe anytime