When AI data leaves the network. Three incidents worth attention.
Every prompt sent to a cloud assistant creates another copy of the document outside the organisation’s direct control. That does not make cloud AI unsafe. It does decide who can inspect the infrastructure afterwards.
The copy may be processed, logged, retained or backed up on infrastructure that other companies operate. The details depend on the provider, the contract, the configuration and the service.
| Incident | Date | What went wrong | Source |
|---|---|---|---|
| Chat & Ask AI database | Reported 9 February 2026 | A Firebase database was publicly readable: 300 million messages, by the researcher’s account | Malwarebytes |
| ChatGPT exfiltration flaw | Patched 20 February 2026 | One malicious prompt could leak messages and uploaded files | The Hacker News, quoting Check Point Research |
| Samsung restriction | May 2023 | Staff used ChatGPT as designed; the reports of secrets typed in are unconfirmed | The Register |
A public database exposed hundreds of millions of messages
On 9 February 2026 Malwarebytes reported that a security researcher had found the backend database of Chat & Ask AI publicly readable. The application is a consumer chatbot built by Codeway, with more than 50 million users. The researcher claims to have accessed 300 million messages, from over 25 million users.
The cause was a Google Firebase project whose security rules allowed the database to be read publicly, which Malwarebytes calls “a well-known and documented Firebase misconfiguration”. The user saw a private conversation. The user could not inspect the configuration and could not verify who reached the infrastructure underneath.
A prompt became a path for data exfiltration
On 30 March 2026 Check Point Research published a flaw in ChatGPT, which OpenAI had patched on 20 February 2026. Check Point put it in one sentence, quoted by The Hacker News: “A single malicious prompt could turn an otherwise ordinary conversation into a covert exfiltration channel, leaking user messages, uploaded files, and other sensitive content.” The same Hacker News article covered a Codex vulnerability, found by BeyondTrust, that exposed GitHub access tokens through a branch name.
Software vulnerabilities are inevitable, including in capable and heavily scrutinised systems. The question for an enterprise is where the affected data sits while the flaw is open. An uploaded document has already crossed the network boundary, and from that point the provider’s controls and response process protect it.
Samsung restricted ChatGPT with no attack at all
Samsung Electronics restricted ChatGPT and similar tools in May 2023. The Register reported the internal memo seen by Bloomberg, which Samsung confirmed to the newswire. Staff were told of “growing concerns about security risks presented by generative AI”.
What staff had typed in is thinner evidence. The Register records unconfirmed reports in Korean media that employees had entered corporate secrets into the chatbot, to fix source code and to generate meeting minutes, and says it asked Samsung to confirm or deny them. On the reported account no attack took place: employees used the tool for its designed purpose, with company information.
What this means for a regulated organisation
A draft contract, a discharge letter, a case file and an internal financial model are all text to the model. The model cannot tell which of them holds special-category personal data under GDPR Article 9, which holds a trade secret and which is safe to share. The bank, the hospital, the insurer or the law firm enforces that boundary, and its security and compliance functions answer these questions:
- Where is the data processed, and which systems store the prompts and the uploaded files?
- How long are they retained, and which subprocessors can reach them?
- In which jurisdictions may the processing take place, and what happens to the backups?
- What happens when the provider changes its architecture?
- What evidence can the organisation produce during an audit?
A contract answers some of these. GDPR Article 28 permits the appointment of a processor, so nothing here is prohibited. What a contract cannot supply is physical control over infrastructure the organisation does not own.
If the data is not sensitive, the use is occasional, and the provider’s contractual and technical controls are acceptable, sending the prompt outside is reasonable. The decision changes when the information itself is the reason the workload cannot sit outside the network.
With Bastion
What Bastion changes
Bastion is a private AI system delivered as one sealed box that runs inside your building. One monthly fee covers the hardware, the model, the hardened operating system and support, and nothing your team types leaves the building.
With Bastion the prompt travels from a workstation to a local server and back, without leaving the organisation’s network. There is no external database to configure, no cloud region to select, and no third-country transfer for that processing path.
A local appliance does not stop an employee uploading the wrong document. It does not prevent excessive privileges, it does not make an answer correct, and it does not replace access control, logging, governance or a data protection assessment. It removes one whole category of external infrastructure from the data path.
Questions this article answers
- Is ChatGPT safe for confidential information?
- Cloud AI is not unsafe, but each prompt creates a copy outside the organisation’s control. On 30 March 2026 Check Point published a ChatGPT flaw, patched on 20 February, that could turn a conversation into an exfiltration channel.
- Is it legal to use cloud AI with personal data?
- Yes. GDPR Article 28 permits a processor. The organisation still answers where the data is processed, who can reach it and for how long.
- Why did Samsung ban ChatGPT?
- In May 2023 Samsung restricted ChatGPT and similar tools, over “growing concerns about security risks presented by generative AI”. The reports that staff had entered corporate secrets are unconfirmed.
On the record
- 1
- 2
The Hacker News
OpenAI Patches ChatGPT Data Exfiltration Flaw and Codex GitHub Token Vulnerabilityread 2026-09-21
- 3