Buy a service or take on operations?
Cloud AI shifts parts of infrastructure, scaling and updates to the provider. Local AI brings control, and more operating responsibility, into the organisation. Decide per workflow and data class, not as an ideology for the whole organisation.
Boundary: control requires operating capability
A local data path is dependable only when access, patching, model and licence status, monitoring, backup and incident ownership are defined. In the cloud, evidence the contract, region, subprocessors, retention, identity and exit route.
Pilot: test quality and operations separately
- Document the same redacted cases and the hardware class
- Measure output quality, latency and throughput separately
- Simulate update, outage, backup and manual fallback
- Compare total cost including internal operating hours
Decision matrix
| Criterion | Cloud AI | local AI |
|---|---|---|
| Data path | Evidence contract, region, retention and subprocessors | Evidence storage, network, access and telemetry internally |
| Operating work | Provider operation plus internal configuration and control | Hardware, models, updates, monitoring and support internally |
| Exit evidence | Test data export, deletion and provider change | Test backup, recovery and component replacement |
Keep it verifiable
Primary sources
- Ollama quickstartSource checked:
- Ollama API authenticationSource checked:
- vLLM OpenAI-compatible serverSource checked:
- Hugging Face model cardsSource checked:
- EU AI ActSource checked:
- EDPB: data protection and AISource checked:
