The moment a regulated team decides to self-host a model
A bank's data science team wants to use a large language model to triage suspicious-transaction alerts. Legal says customer data cannot leave the country, let alone the corporate network. The public API is off the table. So the team does the sensible thing: it downloads an open-weight model, the kind now published on Hugging Face under a permissive licence, and plans to run it inside its own data centre. On paper this solves the data-residency problem in an afternoon.
It also opens a set of problems that most teams do not see until an auditor, or an attacker, points them out. The recent wave of capable open-weight releases has made self-hosting realistic for organisations that could never send data to a third party. A model shipped under Apache 2.0, with a quantised checkpoint that fits on four GPUs, is genuinely usable inside a bank, a telecom core, or an air-gapped industrial network. What comes with it is a supply chain you now own end to end, a model whose safety behaviour anyone can retrain away, and an inference stack that behaves like a hungry, internet-facing application even when you meant it to be neither.
We work with banking, telecom, and critical-national-infrastructure clients who are making exactly this decision right now. This is the security guidance we give them before the first checkpoint is downloaded.
Why "open weight" is not the same as "open" or "safe"
An open-weight model gives you the trained parameters. That is the file you load onto GPUs to run the model. It is not the training data, it is usually not the full training recipe, and it is rarely the training code. Model cards describe data provenance in general terms and little more. You can run the model, fine-tune it, and quantise it. You cannot fully audit what went into it.
That gap matters in a regulated setting for three reasons. Provenance is thin, so you cannot prove to a regulator exactly what corpus shaped the model's outputs. The safety behaviour that ships with the model, the refusals and guardrails, can be removed by anyone who fine-tunes it, so those guardrails are a starting position rather than a control you can rely on. And the weights themselves are a binary artefact downloaded from a public host, which makes them a software-supply-chain component with all the tampering and integrity questions that implies.
None of this makes open-weight models a bad choice. For data-sovereignty-constrained work, self-hosting is often the only defensible option. The point is that the download is the beginning of your security work, not the end of it.
The three risks that actually bite
Model supply-chain integrity
The OWASP Top 10 for LLM Applications (2025) lists supply chain (LLM03) and data-and-model poisoning (LLM04) as distinct, top-tier risks, and for self-hosted deployments they are the ones we see teams underestimate most. A weights file pulled from a public repository can be swapped, back-doored, or subtly poisoned during fine-tuning. Serialization formats matter here: older pickle-based checkpoint formats can execute arbitrary code on load, which turns "download and run a model" into "download and run a stranger's code inside your GPU cluster." Prefer safetensors or an equivalent non-executable format, verify checksums and signatures against the publisher, and treat the model file with the same scrutiny you would apply to any third-party binary entering a production network.
Data leaving the boundary you thought was closed
Teams self-host precisely to keep data inside the perimeter, then wire the model to a vector database, a logging pipeline, and a monitoring agent that all quietly phone home. Sensitive-information disclosure is LLM02 on the OWASP list for a reason. Prompts and retrieved context are data, and every hop they take, into a trace, a cache, an embedding store, an off-site log aggregator, is a place that data can escape. Self-hosting the model while streaming prompts to a cloud observability service defeats the entire purpose.
An inference endpoint that behaves like an open application
Once the model is serving, it is an application endpoint. It accepts untrusted input, it can be prompted into unbounded resource consumption (LLM10), and if it drives tools or agents it can be pushed into actions you did not intend (excessive agency, LLM06). A model that can call internal APIs, run queries, or trigger workflows needs the same authorization boundaries as any service with those privileges, not a blanket trust because "it is our own model."
Choosing a deployment model with your risk appetite in mind
Where and how you host changes the threat model substantially. The table below is the framing we use in early architecture sessions with clients weighing sovereignty against operational cost.
Deployment patternData exposureBest suited toMain security burden
Public hosted API
Data leaves your control; governed by vendor contract
Non-sensitive, public-data workloads
Contractual and DPA controls; output handling
Private cloud (VPC) self-host
Stays in your cloud tenancy
Regulated data with an approved cloud region
Tenancy isolation, key management, egress control
On-premises self-host
Stays in your data centre
Banking core, health records, national data
Full supply-chain and infrastructure ownership
Air-gapped self-host
No external network path
CNI, defence, classified environments
Offline model intake, patching, and update hygiene
The trade-off is consistent: the more control you take over the data, the more of the security stack you own yourself. An air-gapped deployment removes the exfiltration path almost entirely, and in exchange you inherit the whole problem of getting a multi-gigabyte checkpoint, and every subsequent patch, across the gap safely without smuggling something along with it.
A reference architecture for regulated self-hosting
The pattern that holds up under audit puts the model inference layer in its own isolated network segment with no direct route to the public internet. Requests reach it through a gateway that authenticates the caller, enforces rate limits, and logs the request metadata, but not necessarily the raw sensitive payload, to an in-boundary log store. Retrieval, embeddings, and any vector store live inside the same trust boundary. Outbound network access from the inference segment is denied by default and opened only for named, justified destinations.
Key management follows the same discipline you already apply under ISO 27001. Model artefacts, fine-tuning datasets, and any adapters are stored encrypted, with access controlled and logged. Fine-tuning happens in a segregated environment, and the resulting weights are checksummed and version-controlled so you can prove which model answered which request. That provenance record is what turns "we ran an AI model" into "we can show exactly which governed artefact produced this decision," which is the difference an examiner cares about.
For agent-style deployments, where the model can invoke tools, every tool call passes through an authorization layer that checks what this caller is allowed to do, independent of what the model decided to attempt. The model proposes; a separate control decides. This is the single most effective guard against the excessive-agency failure mode, and it is far easier to build in from the start than to retrofit.
Mapping AI controls to frameworks you already run
Most of our clients are not starting from zero. They already hold ISO 27001 certification, run against NIST CSF, or answer to PCI-DSS. Self-hosted AI does not need a parallel universe of controls; it needs your existing controls extended to cover a new class of asset.
ISO/IEC 42001, published as the management-system standard for artificial intelligence, slots alongside ISO 27001 the way a specialised annex would, giving you a governance structure for AI-specific risks without discarding the information-security management system you already operate. The NIST AI Risk Management Framework provides the govern-map-measure-manage cycle for reasoning about AI risk, and it crosswalks cleanly onto NIST CSF functions your security team already knows. The OWASP Top 10 for LLM Applications gives your engineers a concrete, testable checklist at the application layer. For organisations touching the EU market, the EU AI Act adds obligations that scale with the risk class of the system, and those obligations are far cheaper to meet when governance was designed in rather than bolted on after deployment.
The practical move is to treat a self-hosted model as an asset in your existing register, run a risk assessment against it, assign an owner, and pull its specific threats, poisoning, prompt injection, data leakage, into the controls and testing regime you already maintain. AI governance done this way is an extension of good security governance, not a separate programme competing with it.
A pre-deployment checklist
Before an open-weight model serves a single production request in a regulated environment, we want clear answers to the following:
- Provenance and integrity: Where did the weights come from, is the file in a non-executable format, and have you verified checksums and signatures against the publisher?
- Licence fit: Does the licence permit your intended commercial and fine-tuning use, and has legal confirmed it?
- Network isolation: Is the inference layer in its own segment with default-deny egress, and have you proven no telemetry or logging path leaks prompt data outside the boundary?
- Data handling: Are prompts, retrieved context, and outputs classified, and is each storage or trace destination inside the approved boundary?
- Access control: Is every caller authenticated and authorized at a gateway, with rate limiting to bound resource consumption?
- Tool and agent boundaries: If the model can act, does an independent authorization layer gate every tool call?
- Output handling: Is model output treated as untrusted before it reaches a browser, a shell, a database, or another system?
- Guardrails: Since the model's built-in safety can be fine-tuned away, what external moderation and filtering sit around it?
- Monitoring: Are you logging enough to detect prompt-injection attempts, abnormal consumption, and drift, without logging the sensitive payloads themselves?
- Governance: Is the model in your asset register, risk-assessed, owned, and mapped to ISO 27001, ISO 42001, and NIST controls?
- Patch and update path: For on-prem and air-gapped deployments, how do new model versions and security patches arrive, and who verifies them before they land?
- Incident response: If the model is compromised or misused, is it in your IR playbook with a defined containment and rollback step?
A team that can answer these has done the work. A team that downloaded a checkpoint and pointed an application at it has not, however good the demo looked.
How Aydahwa Enterprise Can Help
We help organisations in banking, telecommunications, and critical national infrastructure adopt AI without loosening the controls that keep them compliant. Our background is hands-on: cloud and on-premises architecture, secure network segmentation, and compliance programmes aligned to ISO 27001, PCI-DSS, SOC 2, NIST CSF, and CIS Benchmarks, led by a team that holds the Microsoft Cybersecurity Architect Expert certification and has run security in regulated, high-stakes environments.
For self-hosted and air-gapped AI specifically, we design the isolated inference architecture, the supply-chain verification process for model artefacts, the egress and data-handling controls, and the governance mapping that lets you show an auditor exactly how the deployment is controlled. If you are earlier in the journey, our free cybersecurity self-assessment and readiness checklist are a fast way to see where the gaps are before they reach production.
Explore our cybersecurity services for security architecture and compliance, our cloud services for secure migration and private-cloud deployment, and our managed IT support for ongoing operations. When you are ready to talk through a specific deployment, get in touch and we will start with your data-residency constraints and work outward from there.
