Private AI or cloud AI? Read the provider's own terms first.
A serious cloud AI service makes strong, written commitments about your data. Microsoft's documentation for models sold by Azure states them plainly. It also states the conditions, and the conditions decide the architecture.
Cloud AI and on-premise AI run the same class of model. The difference is where the computation happens and who controls the environment around it.
What a serious cloud service does commit to
Microsoft's documentation for models sold by Azure states that your prompts, completions, embeddings and training data “are NOT available to other customers”, “are NOT available to OpenAI or other providers of Models sold by Azure”, and “are NOT used by providers of Models sold by Azure to improve their models or services”.
On the model itself: “The models are stateless: no prompts or completions are stored in the model. Additionally, prompts and completions are not used to train, retrain, or improve the base models.”
So the claim that cloud AI hands your documents to a model trainer is false for this service.
The conditions: where processing happens and who may read a prompt
The same document sets out where the processing happens, and it depends on a setting. “For any deployment type labeled 'Global,' prompts and responses may be processed in any geography where the relevant model sold by Azure is deployed”. For a deployment labelled DataZone created in an EU member state, processing may happen “in that or any other European Union Member Nation”.
The place where a prompt is processed is therefore a property of the deployment type somebody chose, and somebody who does not know why it was chosen can change it. Data stored at rest stays in the geography the customer designated.
The document also describes abuse monitoring. A sample of prompts and completions may be stored and reviewed. “Human reviewers assessing potential abuse can access prompts and completions data only when that data has already been flagged by the abuse monitoring system, or when the prompts and completions are part of a potentially abusive pattern of use.”
For services deployed in the European Economic Area, “the authorized Microsoft employees are located in the European Economic Area”. Customers who need that turned off have to apply for it: the page describes managed customers who “apply to modify abuse monitoring”.
The question that replaces "is cloud AI secure?"
The useful question is whether this service, in this configuration, provides the controls this data requires, and whether the configuration will still be right in two years.
For most workloads, in our view, the answer is yes and the cloud is the cheaper route. For a contract under negotiation, a patient record, a criminal file or unreleased source code, the answer depends on a deployment-type setting, an approval process and a review queue that the organisation does not run.
What changes with local inference
With a local deployment the prompt, the document and the answer stay inside the network that produced them. The organisation takes on the hardware, the capacity, the updates and the resilience in exchange, and it gives up the three things in the last rows of the table.
| Cloud AI service | Local inference | |
|---|---|---|
| Where processing happens | Depends on the deployment type somebody chose (Microsoft Learn) | Inside the network that produced the prompt |
| Abuse monitoring | A sample of prompts may be stored and reviewed (Microsoft Learn) | No sample and no external queue |
| Largest frontier models | Available | Not available this way |
| Capacity | Elastic | Fixed |
| A model upgrade | Done with a credit card | A planned change |
Which one is cheaper depends on usage
In our view, neither is cheaper for every workload. Cloud is efficient at low and unpredictable usage, because the meter stops. Dedicated infrastructure becomes attractive when usage is steady and high, because the meter does not exist. Compare total cost, usage, control and operational burden over the life of the deployment, not the monthly cloud bill against a hardware price.
With Bastion
What Bastion changes
Bastion is a private AI system delivered as one sealed box that runs inside your building. One monthly fee covers the hardware, the model, the hardened operating system and support, and nothing your team types leaves the building.
Bastion is the second architecture, delivered as one system. The appliance sits at the customer's site or in a data centre and answers on the customer's private network, with no outbound internet connection required, so there is no deployment-type setting that can move the processing and no external review path to document.
The capacity is fixed and stated, which is the trade being made. A Reasoning node serves up to 110 people with thinking off, or 70 with thinking on, at 8k tokens of context, at 30 tokens per second each (p90), with an fp8 KV cache (based on a test on a rented 600 W card with the same chip, before our first box is built).
A workload larger than that is a second node or a different architecture, and it is better to know which before signing anything.
Questions this article answers
- Is cloud AI less secure than on-premise AI?
- Microsoft's documentation for models sold by Azure states that prompts, completions, embeddings and training data are not available to other customers, are not available to the model providers, and are not used to improve their models. The difference is who configures the data boundary.
- Does a cloud AI provider train on company prompts?
- A documented enterprise service says it does not. The same page states that the models are stateless, that no prompts or completions are stored in the model, and that prompts and completions are not used to train, retrain or improve the base models.
- Can a human at the provider read a company prompt?
- Under abuse monitoring, yes, and only under stated conditions: human reviewers can access prompts and completions only when the data has already been flagged by the abuse monitoring system, or when it is part of a potentially abusive pattern of use. For services deployed in the European Economic Area, the authorized Microsoft employees are located there, and a customer who needs the monitoring changed has to apply for it.
On the record
- 1
Microsoft Learn
Data, privacy, and security for Foundry Models sold by Azure in Microsoft Foundryread 2026-11-17