The regulatory pressure that pushed AI back inside the building
Over the last two years, most of the AI questions we get from banks, telecom operators, and critical-infrastructure clients in the UAE have stopped being about what a model can do and started being about where the data goes. A retail bank wants to summarise loan documents. A telco wants an internal assistant that answers policy questions from its own knowledge base. A government-linked operator wants to triage security alerts faster. Every one of those use cases is straightforward with a public API, and every one of them runs straight into the same wall: the data those models would see is regulated, and it is not allowed to leave the country, the tenant, or in some cases the physical premises.
That wall is real. The UAE Personal Data Protection Law (Federal Decree-Law No. 45 of 2021) sets conditions on cross-border transfers of personal data. The Central Bank's regulations expect financial institutions to keep tight control over where customer data is processed. NESA's Information Assurance Standards, which the Signals Intelligence Agency oversees, apply to entities the state considers critical, and for Abu Dhabi's health sector the ADHICS standard adds another layer. Dubai government entities work to the DESC Information Security Regulation. None of these frameworks forbid AI. What they forbid is the casual movement of sensitive data to a processor you do not control, in a jurisdiction you cannot audit.
So the interesting engineering question for regulated sectors is no longer "cloud or on-prem." It is: can we run a capable model entirely inside our own security boundary, and can we secure it well enough that our regulator, our internal audit, and our own risk team sign off? For the first time, the answer is a defensible yes. The rest of this article is about how to earn that yes.
Open-weight models are finally good enough to run yourself
The reason this conversation shifted is supply. A few years ago, self-hosting meant accepting a large capability gap against the frontier hosted models. That gap has narrowed to the point where, for most enterprise tasks, it no longer drives the decision. A short tour of what is available under genuinely usable licences tells the story:
- Llama 4 Scout — Meta's natively multimodal open-weight model, a common starting point because of its tooling and community support.
- DeepSeek V4 — a Mixture-of-Experts model under an MIT licence with a very large context window and near-frontier quality at a fraction of the cost per token.
- Qwen3 — Alibaba's flagship open-weight family under Apache 2.0, with switchable reasoning modes that let you trade latency for depth.
- Gemma 4 — Google's open-weight family, notable for wide language coverage, which matters in a market that operates in Arabic and English side by side.
- Phi 4 and Mistral Small 3.1 — compact models that fit on modest hardware, well suited to on-device or branch-office deployment.
- GLM 5.1, Kimi K2.6, and StarCoder2 — strong on coding and agentic tasks, useful when the internal use case is developer productivity behind the firewall.
- OLMo 2 and Falcon 3 — fully reproducible or lightweight options; Falcon in particular has regional relevance, having come out of the Technology Innovation Institute in Abu Dhabi.
The licences matter as much as the benchmarks. Apache 2.0 and MIT give you the right to run the weights commercially, inside an air-gapped environment, without phoning home. That is a compliance property, not just a legal nicety. A model you can run with no outbound network dependency is a model your data-residency argument can actually rest on.
One distinction worth keeping straight during architecture work is small language models versus large ones. Sub-10B models run on a single GPU or even a capable laptop, respond fast, and cost little to serve, but they lose the thread on long documents and multi-step reasoning. The larger models handle complex reasoning and long context but demand real GPU capacity. In practice we rarely deploy one model for everything. A small model handles classification, routing, and simple lookups at the edge; a larger model sits centrally for the harder work. That split is also a security control, because it limits how much sensitive context any single component ever holds.
A reference architecture for on-premises LLM deployment
A self-hosted LLM is not one thing. It is a small stack, and each layer has its own failure modes. When we design these for clients, we treat it as three tiers: the inference layer that serves the model, the data layer that feeds it, and the access layer that decides who and what may talk to it.
The inference layer
The serving engine is where most teams start and where a surprising number of mistakes get baked in. vLLM and Hugging Face's Text Generation Inference are the usual choices for GPU-backed production serving; Ollama and llama.cpp are lighter options for smaller footprints and edge nodes. Whichever you pick, the security-relevant decisions are the same. The inference server should never be exposed directly to a user network. It sits behind an application gateway that handles authentication, rate limiting, request logging, and input and output filtering. The model weights live on encrypted storage. The container image is built from a pinned base, scanned before deployment, and hardened against a CIS Benchmark. GPU nodes are patched on the same cadence as the rest of your estate, which sounds obvious until you meet the AI cluster that nobody in IT operations knew existed.
Network segmentation and the air-gap question
Clients often ask for a fully air-gapped deployment and then discover it costs them more than the risk warrants. A true air gap, with no network path in or out, is appropriate for the most sensitive classified or critical-infrastructure workloads, and we have built them. For most regulated commercial use, a segmented deployment is the right balance: the model and its data sit in a dedicated network zone with no outbound internet access, all traffic to it passes through an inspected, authenticated gateway, and model updates arrive through a controlled, reviewed pipeline rather than a live download. The test to apply is simple. If someone unplugged your internet uplink, would the assistant still work and would any sensitive data still be safe? If yes, your residency and exfiltration story holds up under questioning.
Retrieval and data governance
Almost every real deployment adds retrieval, so the model can answer from company data it was never trained on. This is where governance gets missed. The vector database that stores your embeddings holds a searchable copy of your source documents, and it inherits every access-control obligation the originals had. If a user cannot see a document in SharePoint, the assistant must not surface its contents through retrieval either. That means access control has to be enforced at query time, filtered by the identity of the person asking, not bolted on afterwards. We have seen more than one proof-of-concept quietly become a data-leak channel because retrieval ignored the permissions that the underlying repository spent years getting right.
The threat model is different from a cloud API
When you host the model yourself, you inherit categories of risk that a cloud provider used to absorb on your behalf. The OWASP Top 10 for LLM Applications (2025) is a good shared vocabulary for these, and it maps cleanly onto what an on-premises operator has to own. A few of the categories deserve specific attention in a self-hosted context.
Supply chain and model poisoning. You are now downloading multi-gigabyte weight files and running them inside your trust boundary. Older checkpoint formats based on Python pickle can execute arbitrary code on load, which makes an untrusted weight file a genuine remote-code-execution risk. Prefer the safetensors format, verify checksums against the publisher, and treat a model artifact with the same suspicion you would treat any third-party binary. A model registry with provenance records, the AI equivalent of a software bill of materials, is not paperwork for its own sake. It is what lets you answer "where did this weight file come from and who approved it" during an incident.
Prompt injection and excessive agency. The moment your assistant can call tools, read email, or query systems, a cleverly crafted document can hijack it into doing something the user never asked for. The defence is old-fashioned least privilege applied to a new actor. The model gets its own service identity, scoped permissions, and an approval step for anything consequential. Give an agent the ability to plan and execute on its own and you have created a non-human user with standing access, so govern it like one.
Sensitive information disclosure and output handling. Log what goes in and what comes out, because you will need those logs for audit and for incident response, but recognise that the logs themselves now contain sensitive prompts and must be protected accordingly. Filter model output before it reaches downstream systems; treating generated text as trusted input to a database query or a shell is how injection turns into compromise.
Cloud API versus self-hosted, without the marketing
Self-hosting is not automatically more secure. It moves the responsibility to you, which is better only if you actually discharge it. The honest comparison looks like this:
DimensionHosted API (public cloud model)Self-hosted open-weight model
Data residency
Depends on provider region and contract; cross-border transfer risk
Fully within your boundary and jurisdiction
Time to first value
Days
Weeks to months
Security responsibility
Shared; provider owns the model plane
Entirely yours, top to bottom
Cost profile
Per-token operating expense, scales with usage
Upfront GPU capital, cheaper at steady high volume
Capability ceiling
Frontier
Near-frontier for most enterprise tasks
Supply-chain exposure
Absorbed by provider
You load and vet the weights
Audit and explainability
Limited visibility into the model plane
Full visibility; you control logging and versions
For a low-sensitivity internal tool, a hosted API in an approved region is often the pragmatic choice, and we say so. The self-hosted path earns its cost when the data is regulated, the volume is high and steady, or the regulator simply will not accept the data leaving your control. Most of our banking and critical-infrastructure clients land in that second group, which is why the question keeps coming up.
Mapping controls to the standards your auditor already uses
One thing that makes an on-premises AI project defensible is that you do not need a brand-new control framework for it. The obligations map onto standards your organisation is very likely already measured against. NIST CSF 2.0 gives you the govern-identify-protect-detect-respond-recover structure to organise the whole programme. ISO/IEC 27001 covers the information-security management system the deployment lives inside, and ISO/IEC 42001, the AI management system standard, extends that to AI-specific governance if you want a certifiable target. CIS Benchmarks harden the hosts and containers. If cardholder data ever enters a prompt or a retrieval corpus, PCI-DSS scope follows it, and the model environment becomes part of your cardholder data environment whether you planned for that or not. The NIST AI Risk Management Framework rounds it out with a vocabulary for the model-specific risks that traditional infosec frameworks were not written to address.
The practical value of this mapping is speed. When we frame a self-hosted LLM as a set of assets, data flows, and controls expressed in the language of ISO 27001 and NIST CSF, an internal audit team can review it against processes they already run. The AI stops being a special exception that nobody knows how to sign off, and becomes another in-scope system with owners, controls, and evidence.
A deployment checklist from our engagements
This is the short version of what we walk through before an on-premises model goes anywhere near production data:
- Classify the data the model and its retrieval corpus will touch, and confirm what the relevant regulation permits for it.
- Choose models by licence first, then capability, and record the licence terms for each.
- Verify every weight file: checksums against the publisher, safetensors over pickle, provenance logged in a model registry.
- Place inference nodes in a dedicated segment with no outbound internet path and no direct user access.
- Put an authenticated gateway in front of the model for identity, rate limiting, input and output filtering, and logging.
- Enforce retrieval access control at query time, scoped to the requesting user's real permissions.
- Give any agent its own scoped service identity and require approval for consequential actions.
- Harden hosts and containers to a CIS Benchmark and bring GPU nodes into your normal patch cycle.
- Protect the prompt and completion logs as the sensitive data they now contain.
- Map every control to NIST CSF and ISO 27001, and red-team the deployment against the OWASP LLM Top 10 before go-live.
None of these steps is exotic. They are the same disciplines that secure any sensitive system, applied to a component that happens to reason in natural language. The failures we get called in to fix almost never come from an unbeatable new attack. They come from a proof-of-concept that skipped the boring parts and went straight to production.
How Aydahwa Enterprise Can Help
Aydahwa Enterprise designs and secures on-premises and hybrid AI infrastructure for organisations that cannot send their data to someone else's cloud. Our work is grounded in 25 years of hands-on infrastructure and security architecture across banking, telecom, and critical national infrastructure, and in the standards regulators actually check against, including ISO 27001, PCI-DSS, SOC 2, NIST CSF, the NIST AI RMF, and CIS Benchmarks, backed by credentials such as the Microsoft Cybersecurity Architect Expert certification.
If you are weighing a self-hosted model, we can help you scope the architecture, choose the right models and serving stack, segment the network, build the governance and control mapping your auditor will expect, and red-team the result before it touches real data. Start with our cybersecurity services and cloud security and migration practices, or lean on our managed IT support to run the environment once it is live. To gauge where you stand today, run the free cybersecurity self-assessment and work through our cybersecurity readiness checklist. When you are ready to talk specifics, get in touch and we will map your use case to a deployment your risk team can sign off.
