Skip to content
All articles
Private AI

Seven questions to answer before you decide where AI runs

The place where AI runs follows from the workload: what the system reads, how steadily people use it, and what happens when it stops.

BastionUpdated 4 min read

Most on-premise decisions start with a card and a price. The card is the last question. These seven come before it, and the answers usually decide the architecture without an argument.

1. What will the system read?

List what it will read, not what it is for. If the answer is public material, marketing copy and open documentation, a cloud service is appropriate and cheaper. If the answer includes client matter files, patient records, salary data, pricing models or unreleased source code, the architecture needs a closer look.

2. Does the data have to leave the organisation?

Microsoft's documentation for models sold by Azure states that “For any deployment type labeled 'Global,' prompts and responses may be processed in any geography where the relevant model sold by Azure is deployed”, while for a DataZone deployment created in an EU member state, processing may happen “in that or any other European Union Member Nation”.

Residency is a setting that somebody chooses and somebody else can change. If nothing requires the data to leave, a local deployment removes the setting.

3. How steady is the usage?

Cloud is at its best when demand is spiky and unpredictable, because the meter stops when nobody is working. Thirty people using AI every working day for similar tasks is the opposite shape, and it is the shape that makes dedicated capacity easy to evaluate. Estimate concurrent users, not employees: a hundred-person company rarely has a hundred people waiting on a model at the same second.

4. Does the AI need the internet?

Many internal tasks do not. Document search, summarisation, drafting, translation and internal question answering can all work from data that is already inside.

Removing connectivity a workload does not need removes exposure. ENISA's Threat Landscape 2025, published on 1 October 2025, analysed “4875 incidents over a period spanning from 1 July 2024 to 30 June 2025”, and recorded that “By early 2025, AI-supported phishing campaigns reportedly represented more than 80 percent of observed social engineering activity worldwide.”

Juhan Lepassaar, ENISA's executive director, framed the wider problem: “Systems and services that we rely on in our daily lives are intertwined, so a disruption on one end can have a ripple effect across the supply chain.”

5. How strong does the model have to be?

If the work needs the strongest frontier reasoning available this quarter, the cloud has it first and an appliance does not. If the work is summarising a contract, answering a question from internal policy, drafting a reply or extracting clauses, a well-served open-weight model is sufficient, and the difference is not visible in the output.

6. Who operates it?

On-premise does not mean unattended. Updates, access, backups, monitoring and incident response belong to somebody. An appliance reduces that burden to a supported update path. A self-assembled stack does not.

7. What happens when it is unavailable?

Once a workflow depends on AI, availability is an operational property. However reliable the provider, the organisation needs to know what it does during the hours when the answer does not come back.

The AI Act applies wherever the model runs

The European Commission's timeline records that the AI Act “entered into force on 1 August 2024 and became applicable on 2 August 2026”, that “prohibited AI practices and AI literacy obligations entered into application from 2 February 2025”, and that “the rules for high-risk use cases in certain sensitive areas (Annex III) have been extended to 2 December 2027”. Under Regulation (EU) 2024/1689, the classification follows the intended purpose of the system (Article 6) and an obligation follows the role of the organisation, provider or deployer (Article 3), so moving the server does not move the classification.

With Bastion

What Bastion changes

Bastion is a private AI system delivered as one sealed box that runs inside your building. One monthly fee covers the hardware, the model, the hardened operating system and support, and nothing your team types leaves the building.

Bastion exists for the answer set where questions 1, 2, 4 and 7 point inwards and question 5 does not demand a frontier model. That is a subset of workloads.

The sizing conversation starts from the same place as this article: what the system will read, how many people will use it at once, and what the work is. Head count alone does not size a box.

Questions this article answers

On-premise AI or cloud: how do I decide?
By the workload, not the hardware. What the system will read, whether the data has to leave, how steady the usage is, whether the internet is needed, how strong the model must be, who operates it, and what happens when it is unavailable.
Is cloud AI processing always in my country?
Not automatically. Microsoft's documentation for models sold by Azure states that for any deployment type labelled Global, prompts and responses may be processed in any geography where the model is deployed. Residency is a deployment setting, not a property of the service.
Does moving AI on-premise change our AI Act obligations?
No. An obligation follows the system's intended purpose and the organisation's role. The Commission records that the Act became applicable on 2 August 2026, with the Annex III high-risk rules extended to 2 December 2027.

On the record

  1. 1
  2. 2
  3. 3

    European Commission

    AI Act. Regulatory framework for AI

    read 2026-09-22

  4. 4