Most on-premise decisions start in the wrong place, with a card and a price. The card is the last question, not the first. These seven come before it, and answering them honestly usually decides the architecture without an argument.
1. What will the system actually read?
Not what it is for. What it will read. If the answer is public material, marketing copy and open documentation, a cloud service is appropriate and cheaper. If the answer includes client matter files, patient records, salary data, pricing models or unreleased source code, the architecture deserves examination.
2. Does the data have to leave the organisation?
This question has a documented answer, not an assumed one. Microsoft's documentation for models sold by Azure states that “For any deployment type labeled 'Global,' prompts and responses may be processed in any geography where the relevant model sold by Azure is deployed”, while for a DataZone deployment created in an EU member state, processing may happen “in that or any other European Union Member Nation”.
That is not a criticism of the service. It is a demonstration that residency is a setting somebody chooses, and a setting somebody else can change. If nothing requires the data to leave, a local deployment removes the setting rather than configuring it.
3. How steady is the usage?
Cloud is at its best when demand is spiky and unpredictable, because the meter stops when nobody is working. Thirty people using AI every working day for similar tasks is the opposite shape, and it is the shape that makes dedicated capacity easy to evaluate.
Estimate concurrent users, not employees. A hundred-person company rarely has a hundred people waiting on a model at the same second.
4. Does the AI need the internet?
Many internal tasks do not. Document search, summarisation, drafting, translation and internal question answering can all work from data that is already inside.
Removing connectivity a workload does not need removes exposure. ENISA's Threat Landscape 2025, published on 1 October 2025, analysed “4875 incidents over a period spanning from 1 July 2024 to 30 June 2025”, and recorded that “By early 2025, AI-supported phishing campaigns reportedly represented more than 80 percent of observed social engineering activity worldwide.” Juhan Lepassaar, ENISA's executive director, framed the wider problem: “Systems and services that we rely on in our daily lives are intertwined, so a disruption on one end can have a ripple effect across the supply chain.”
5. How strong does the model have to be?
This is where the honest answer often favours the cloud. If the work needs the strongest frontier reasoning available this quarter, the cloud has it first and an appliance does not.
If the work is summarising a contract, answering a question from internal policy, drafting a reply or extracting clauses, a well-served open-weight model is sufficient, and the difference is not visible in the output.
6. Who operates it?
On-premise does not mean unattended. Updates, access, backups, monitoring and incident response belong to somebody. An appliance reduces that burden to a supported update path. A self-assembled stack does not.
7. What happens when it is unavailable?
Once a workflow depends on AI, availability stops being a convenience and becomes an operational property. The question is not whether the provider is reliable. It is what the organisation does during the hours when the answer does not come back.
The decision is not cloud against on-premise. It is which architecture fits this workload, and the workload has already answered most of it.
The question that is not on the list
Where the model runs does not change what the law calls it. The European Commission's timeline records that the AI Act “entered into force on 1 August 2024 and became applicable on 2 August 2026”, that “prohibited AI practices and AI literacy obligations entered into application from 2 February 2025”, and that “the rules for high-risk use cases in certain sensitive areas (Annex III) have been extended to 2 December 2027”.
An obligation follows the purpose of the system and the role of the organisation. Moving the server does not move the classification.
With Bastion
What Bastion changes
Bastion is a private AI system delivered as one sealed appliance that runs inside your building. One monthly fee covers the hardware, the model, the hardened operating system and support, and nothing your team types leaves the building.
Bastion exists for the answer set where questions 1, 2, 4 and 7 point inwards and question 5 does not demand a frontier model. That is a real subset of workloads, not all of them.
The sizing conversation starts from the same place as this article: what the system will read, how many people will use it at once, and what the work is. Head count alone does not size a box.
Questions this article answers
- On-premise AI or cloud: how do I decide?
- By the workload, not the hardware. What the system will read, whether the data has to leave, how steady the usage is, whether the internet is needed, how strong the model must be, who operates it, and what happens when it is unavailable.
- Is cloud AI processing always in my country?
- Not automatically. Microsoft's documentation for models sold by Azure states that for any deployment type labelled Global, prompts and responses may be processed in any geography where the model is deployed. Residency is a deployment setting, not a property of the service.
- Does moving AI on-premise change our AI Act obligations?
- No. An obligation follows the system's intended purpose and the organisation's role. The Commission records that the Act became applicable on 2 August 2026, with the Annex III high-risk rules extended to 2 December 2027.
On the record
- 1
Microsoft Learn
Data, privacy, and security for Foundry Models sold by Azure in Microsoft Foundryread 2026-09-22
- 2
- 3
Read next
- What is private AI, and when does an organisation actually need it?Private AI is not a different kind of model. It is a different answer to one question: who controls the environment where the data is processed. What that answer changes, what it leaves untouched, and the five questions that settle it.Read the article
- How to evaluate an AI deployment: the checklist for a 5 to 100 person organisationTen questions about your own work, then four lists to put to any supplier: the technical specification, the data flow, the security architecture and the commercial terms. If a supplier cannot answer them, that is the answer.Read the article