Private AI or cloud AI? Read the provider's own terms first.

The debate is normally presented as cloud AI against on-premise AI, as though one were safe and the other were not. Both run the same class of model. The difference is where the computation happens and who controls the environment around it. The way to see that difference is to read what the cloud provider actually commits to, in its own documentation, rather than what either side says about it in a slide.

What a serious cloud service does commit to

Microsoft's documentation for models sold by Azure is specific. It states that your prompts, completions, embeddings and training data “are NOT available to other customers”, “are NOT available to OpenAI or other providers of Models sold by Azure”, and “are NOT used by providers of Models sold by Azure to improve their models or services”.

On the model itself it is equally direct: “The models are stateless: no prompts or completions are stored in the model. Additionally, prompts and completions are not used to train, retrain, or improve the base models.”

That is a real commitment in a real contract. Anyone who claims that using cloud AI means handing your documents to a model trainer has not read the page. The cloud's case has to be conceded before the rest of the argument is worth anything.

Now read the conditions

The same document sets out where the processing happens, and it depends on a setting. “For any deployment type labeled 'Global,' prompts and responses may be processed in any geography where the relevant model sold by Azure is deployed.” For a deployment labelled DataZone created in an EU member state, processing may happen “in that or any other European Union Member Nation”.

So data residency is not a property of the service. It is a property of the deployment type somebody chose, and it can be changed by somebody who does not know why it was chosen.

The document also describes abuse monitoring. A sample of prompts and completions may be stored and reviewed. “Human reviewers assessing potential abuse can access prompts and completions data only when that data has already been flagged by the abuse monitoring system, or when the prompts and completions are part of a potentially abusive pattern of use.” For services deployed in the European Economic Area, “the authorized Microsoft employees are located in the European Economic Area”. Customers who need that turned off have to apply for it: the page describes managed customers who “apply to modify abuse monitoring”.

None of this is a scandal. It is a well-documented, well-governed system. It is also a system whose data boundary the buyer configures rather than owns.

The question that replaces "is cloud AI secure?"

The useful question is narrower and harder. Does this particular service, in this particular configuration, provide the controls this particular data requires, and will the configuration still be right in two years?

For most workloads the answer is yes, and the cloud is the cheaper route. For a contract under negotiation, a patient record, a criminal file or unreleased source code, the answer depends on a deployment-type setting, an approval process and a review queue that the organisation does not run.

What changes with local inference

With a local deployment the prompt, the document and the answer stay inside the network that produced them. There is no deployment type to get wrong, no abuse-monitoring sample, and no external queue. The organisation takes on the hardware, the capacity, the updates and the resilience in exchange.

It also gives up something real. The largest frontier models are not available this way, capacity is fixed rather than elastic, and a model that can be upgraded with a credit card in the cloud is a planned change on-premise.

Which one is cheaper

Neither, universally. Cloud is efficient at low and unpredictable usage, because the meter stops. Dedicated infrastructure becomes attractive when usage is steady and high, because the meter does not exist. The honest comparison is not the monthly cloud bill against a hardware price. It is total cost, usage, control and operational burden, over the life of the deployment.

Architecture follows the workload. It does not follow a preference.