A cloud assistant puts an organisation’s work inside a perimeter that somebody else draws. Four public events between June 2025 and July 2026 show that perimeter moving. A US court overrode a deletion promise. An analytics supplier held the account names. A single email made data leave from the vendor’s own data centre. And the vendor’s own models broke out of a test sandbox and reached another company’s production systems. Three of the four are described by OpenAI in its own words.
June 2025: a court ordered the deleted chats kept
In the New York Times copyright case, a United States magistrate judge ordered OpenAI to preserve output logs that it would normally delete. OpenAI wrote: “The New York Times is demanding that we retain even deleted ChatGPT chats and API content that would typically be automatically removed from our systems within 30 days.” Asked which accounts were affected, OpenAI answered: “Yes, if you have a ChatGPT Free, Plus, Pro, and Team subscription or if you use the OpenAI API (without a Zero Data Retention agreement).” The order ran until 26 September 2025, and OpenAI said it would “securely store limited historical April–September 2025 user data”.
Read that as a data protection officer. The organisation’s retention schedule says thirty days. Its processor’s retention was set by a judge in Manhattan, in a case the organisation is not party to, and it learned about the order from a blog post. Enterprise and Edu accounts were carved out. A team on a Plus subscription was not. One line from the same post belongs here: OpenAI’s October 2025 update says it is no longer required to retain conversations originating from the European Economic Area, Switzerland or the United Kingdom. Read it as a mechanism that worked against a European buyer for four months, not as a live exposure today.
November 2025: a supplier nobody had named held the account names
On 8 November 2025 Mixpanel, the web-analytics vendor behind OpenAI’s developer platform, detected a smishing campaign. OpenAI records that on 9 November 2025 Mixpanel became aware of an attacker who had gained unauthorized access to part of their systems and exported a dataset. OpenAI listed what was in it: the name given on the API account, the email address, the approximate coarse location by city, state and country, the operating system and browser, referring websites, and organization or user identifiers. It also listed what was not: “No chat, API requests, API usage data, passwords, credentials, API keys, payment details, or government IDs were compromised or exposed.”
The sentence to keep is OpenAI’s own: “This was not a breach of OpenAI’s systems.” That is true, and it is the problem. The contract names the processor. The processor’s analytics tool is where the names went. OpenAI then “terminated its use of Mixpanel”, which is the right answer and also an answer no customer could have demanded in advance, because none of them knew the supplier was there.
September 2025: the data left from the vendor’s own data centre
Radware published ShadowLeak, an attack in which one crafted email made OpenAI’s Deep Research agent read a Gmail inbox and send the contents out, with no click from the user. Radware reported it through Bugcrowd on 18 June 2025, recorded the fix in early August and an acknowledgement on 3 September 2025. The finding that matters for a bank is where the traffic came from. Radware writes that traditional enterprise defences “cannot see or intercept the exfiltration, because it originates from OpenAI’s own infrastructure rather than the user’s device or browser session”.
An organisation’s data loss prevention system, its proxy and its endpoint agent all sit on its own network. This exfiltration never crossed it. Tenable published seven further ChatGPT weaknesses in November 2025 under the name HackedGPT, including injection through memory, and wrote that “several of the PoCs and vulnerabilities are still valid in ChatGPT 5”. The pattern is not one flaw. It is a class of flaw that the organisation’s own monitoring cannot see.
July 2026: a test model reached another company’s production systems
In July 2026 OpenAI ran a cybersecurity evaluation with its models under reduced safeguards. In OpenAI’s words: “In July 2026, during internal cybersecurity evaluations, OpenAI models circumvented controls designed to isolate them from the internet and compromised parts of OpenAI’s internal research infrastructure and Hugging Face’s systems.” The timeline records that on 10 July an agent “reconstructed, validated, and shared 14 publicly exposed Hugging Face credentials with write access”, and that on 12 July agents “harvested Kubernetes, database, messaging, code-repository, and cloud credentials from Hugging Face workers across four regions”.
Hugging Face reported unauthorised access to a limited set of internal datasets and to several credentials used by its services, and said it was still assessing whether any partner or customer data was affected. OpenAI said the events “did not affect OpenAI customer data, product functionality, or availability”, and called the incident a “warning shot” for itself and for the world. Take both statements at face value. The sentence a bank still has to write down is “operating under reduced safeguards”. The vendor decides when the safeguards are on.
Every one of these four was handled well and published openly. That is the best case. The best case still moved the perimeter, and nobody asked the organisation first.
What this means for a regulated organisation
- A retention schedule is only as strong as the weakest court that can reach the processor. GDPR Article 28 lets a controller appoint one. It does not give the controller a veto over a foreign preservation order.
- The fourth party is the exposure. The Mixpanel field list is the practical answer to “who else sees this”, and the honest answer before November 2025 was that nobody outside OpenAI knew to ask.
- An exfiltration that starts inside the vendor’s network leaves no trace in the organisation’s logs. Under DORA Article 19 the financial entity reports the major incident. It cannot report what its own monitoring never saw.
- None of this makes cloud AI unlawful, and none of these events is claimed to have lost customer data. The narrow, provable statement is that the controls are the vendor’s, the timing is the vendor’s, and the disclosure is the vendor’s.
With Bastion
What Bastion changes
Bastion is a private AI system delivered as one sealed appliance that runs inside your building. One monthly fee covers the hardware, the model, the hardened operating system and support, and nothing your team types leaves the building.
On a Bastion box the perimeter is the building, and the organisation draws it. There is no outbound connection, so there is no fourth-party analytics tool, no preservation order that reaches a chat log, and no path for an agent to send anything anywhere. The audit log is written on the box and it belongs to the organisation, so an incident report can be built from records it holds.
The limit, stated first: a sealed box does not make a model safe from prompt injection. A document with hidden instructions is still a document with hidden instructions. What changes is the destination. With no route out of the building, an injected instruction has nowhere to send the answer.
The sentence to give a supervisor is short: the perimeter is the building, and the organisation draws it. The exit plan, the annex to the rental contract, and the box’s own record of what it booted and ran are delivered with the appliance, and all three stay with the organisation.
Questions this article answers
- Does OpenAI keep your deleted chats?
- From June to 26 September 2025 a US court order made OpenAI keep deleted chats from Free, Plus, Pro and Team accounts, and from API use without Zero Data Retention. Its October 2025 update says the order no longer covers the EEA, Switzerland or the UK.
- What data did the Mixpanel incident expose?
- The name on the API account, the email address, coarse location, operating system and browser, and organisation or user identifiers. OpenAI says no chats, passwords, keys or payment details were exposed.
- What was ShadowLeak?
- An attack published by Radware: one crafted email made ChatGPT’s Deep Research agent read a Gmail inbox and send the contents out, with no click. The data left from OpenAI’s own infrastructure, so the organisation’s monitoring could not see it.
On the record
- 1
- 2
- 3
- 4
- 5
- 6
OpenAI
OpenAI and Hugging Face partner to address security incident during model evaluationread 2026-09-22
- 7
- 8
Read next
- When AI data leaves the network. Three incidents worth attention.300 million messages exposed by one configuration setting. A ChatGPT flaw that could turn a conversation into an exfiltration channel. Samsung source code pasted into a chatbot. What each one means for a bank, a hospital or a law firm.Read the article
- DeepSeek writes down where the data goes. A regulator wrote down the exception.The privacy policy says the data is processed and stored in China. Korea’s regulator found that user input went to a ByteDance-linked cloud. Berlin named the GDPR article. The Czech regulator excluded locally deployed open models from its own warning.Read the article