An open-weight model is a set of weights you can download and run. A hosted model is an API you pay per token. The decision is not ideological. It is residency, cost curve, quality on your eval set, and who is awake when it breaks.
We run both. Kitapix catalog tagging and Polylingo draft translations can live on a hosted model because the text is not a child’s raw homework and the quality bar is high. Anything that must stay in the EU or on a customer VPC — or that we will call tens of millions of times — gets a self-hosted candidate on the same eval suite. If the hosted model wins by a point and costs 20×, we still take the open-weight model and spend the week on retrieval instead.
The spreadsheet we actually use
Columns: task, data class (public / personal / child), monthly volume, p95 latency budget, whether we need tool calling, whether we need JSON schema, and whether the customer’s contract forbids a US subprocessors list. We score two hosted models and one open-weight baseline. The winner is the cheapest that clears the quality floor, not the one that wrote the cleverest demo.
Operations are part of quality
A self-hosted 70B that we cannot roll back is worse than a hosted small model with a pinable version. We pin versions in config, keep the previous image, and treat a model upgrade like a database migration: expand, verify, contract. Hallucination regressions are release blockers. “New weights dropped on Tuesday” is not a reason to deploy on Wednesday.
.NET does not have to speak Python
Inference stays in a small Python or C++ sidecar if the runtime wants it. The product stays C#. The contract is HTTP or gRPC with a schema. That keeps hiring inside the studio and keeps the web nodes boring. We have no interest in turning every ASP.NET service into a research cluster.
Questions we keep getting
Is open-weight always cheaper? No. Idle GPUs are expensive. Below a few million tokens a month, an API usually wins. Above that, do the spreadsheet.
What about licenses? We read them. Some weights are open to run and closed to train a competitor. Legal signs the row before we ship.
Can a customer force hosted-only? Yes. Two of our B2B contracts do. The architecture has to allow that swap without a rewrite.