Welcome back. This is Part 2 of the series. In Part 1, I defined an open weight model as a set of trained parameters you hold rather than a capability you rent. Here I want to put open weight and closed models side by side — and puncture a few myths on both sides along the way.
The short version: this is not a quality decision, it is a control decision. Closed models still lead at the very top end of capability. Open weight models win on control, customization and predictable unit economics at scale. Most serious enterprises will need both.
The real difference is who owns which layer
When you call a closed model through an API, your responsibility effectively ends at your application, your prompts and your guardrails. Everything below that — serving infrastructure, GPU capacity, model versioning, where inference physically happens — belongs to the provider. You manage it through contracts, data processing agreements and SOC 2 reports.
Self-host an open weight model and that line drops to the floor. You now own the serving stack, the capacity plan, the version lifecycle and the security of the weights themselves. That is the source of every advantage open weights offer — and every cost.
Who owns each layer: closed API vs. self-hosted open weights
Side by side
| Dimension | Closed (API) | Open weight (self-hosted) |
|---|---|---|
| Peak capability | Leads on hardest reasoning, long-context and agentic tasks | At or near parity on code and many structured tasks; trails on broad knowledge |
| Time to value | Minutes — an API key | Weeks — platform, GPUs and MLOps skills required |
| Cost model | Pay per token; no fixed cost; can spike with volume | Fixed GPU cost; very low marginal cost at high, steady utilization |
| Customization | Prompting, RAG, limited vendor fine-tuning | Full fine-tuning, distillation, quantization, adapters |
| Data residency | Provider’s regions and terms | Wherever you run it, including air-gapped |
| Version control | Provider deprecates on its schedule | You pin a version for as long as you need |
| Security responsibility | Provider secures model and infra; your data leaves your boundary | Data stays in; you own model supply chain, patching and isolation |
| Auditability | Limited to vendor documentation | Full access to weights, activations and logs |
| Lock-in | High — prompts and evals tuned to one vendor | Low at the model layer; shifts to your platform |
Three myths worth retiring
Myth 1: “Open weight is free.” The weights are free. The GPUs, the inference engineers, the observability stack and the on-call rotation are not. A self-hosted model running at 15% GPU utilization can easily cost more per token than a frontier API. The economics only work when you have sustained, predictable volume — or when a requirement like data residency takes the API off the table entirely.
Myth 2: “Closed models are less secure.” Not inherently. The large providers run mature security programs, and many enterprises cannot match them. What closed models cannot offer is your control boundary — your data leaves your environment, and you rely on attestations. Conversely, open weights introduce their own supply-chain risk: tampered checkpoints, unsafe serialization formats, and fine-tuned variants on public hubs with unknown provenance. Owning the stack means owning those risks.
Myth 3: “Open weight models are a generation behind.” That was true in 2023. In 2026 the gap is task-specific. On code generation and many structured tasks the best open models are at or near parity; on graduate-level science, broad world knowledge and long-context retrieval they still trail by a meaningful margin (Digital Applied). The right question is not “which is better?” but “good enough for this workload?”
What this means for the architect
The decision is rarely binary, so stop framing it that way. I advise clients to ask four questions per workload:
- Does the data have to stay inside our boundary? If yes, open weights (or a dedicated, sovereign deployment of a closed model) move to the front of the line.
- Is this the hardest 10% of reasoning, or the routine 90%? Frontier problems favor closed models; high-volume routine work favors open ones.
- Is the volume high and steady enough to keep GPUs busy? If not, per-token pricing is probably cheaper.
- Do we need to freeze, inspect or deeply customize the model? Regulated workloads and domain-specific tasks often do.
In Part 3, I’ll turn those questions into a workload-placement framework and a reference architecture that lets you run both kinds of model behind one governed platform.
Featured image designed by Magnific