The choice between self-hosted vs API open-weight models is not a setup decision you make once — it's an operational parameter you should revisit every six to twelve months as token volumes grow, model generations shift, and both hardware and API pricing curves move in real time. Teams that locked in their serving strategy in 2023 are operating on economics that are now materially wrong.

One critical framing note before the numbers: this analysis applies exclusively to open-weight models — Llama 3.1, DeepSeek R1, Qwen 3, Kimi K3, and their successors. The question is not whether to use a proprietary frontier model versus an open-weight one. The model family is fixed. The only variable is whether you serve it yourself or consume it through a managed endpoint.

Why This Decision Is Urgent Right Now

The open-weight tier is no longer a performance compromise. DeepSeek R1 (671B MoE, January 2025) matched or exceeded proprietary model performance on several reasoning benchmarks. Kimi K3 from Moonshot AI — now available on Amazon Bedrock with a 1-million-token context window and native vision — represents the same shift: frontier-class capability with open weights you can run independently. Bonsai 2 27B, a ternary-weight model built on Qwen architecture, delivers approximately 98% of frontier performance in a 5.9GB footprint runnable on consumer hardware. The "open-weight as second-tier" assumption that historically justified defaulting to proprietary APIs no longer holds.

Meanwhile, both cost curves are moving simultaneously. DeepSeek-V3 entered the market at roughly $0.27 per million tokens, forcing Together AI, Fireworks AI, and Groq to compress margins across the board. At the same time, NVIDIA B100/GB200 availability in H2 2025 reduced self-hosting total cost of ownership by approximately 40% versus H100-era infrastructure. Last year's break-even math is obsolete. So is the year before that.

Regulatory pressure adds a non-economic forcing function that overrides the cost calculation entirely for some teams. The EU AI Act, effective August 2026, alongside sector-specific requirements in healthcare and finance, is pushing data-sensitive workloads toward self-hosting regardless of where the token economics land.

Option A: Managed API (Together AI, AWS Bedrock, Fireworks AI, Groq)

The API route trades per-token cost for operational simplicity. Llama 3.1 70B runs at approximately $0.59–0.90 per million tokens across Together AI and AWS Bedrock. Groq's LPU hardware delivers 500+ tokens per second on 70B-class models at competitive rates. At 10 million tokens per month, your total spend lands in the $600–$900 range with zero capital expenditure, no DevOps headcount, and automatic model version management.

The operational ceiling is the problem. Cost scales linearly with volume — there are no economies of scale above committed-use discounts. At 200 million tokens per month, you're writing a $12,000–$18,000 monthly check for a model whose weights you could run yourself. Data leaves your perimeter on every inference call, with compliance posture varying by provider. Network overhead adds 100–400ms per request, which is acceptable for asynchronous document processing but disqualifying for real-time voice or low-latency agent pipelines. And when a provider deprecates an endpoint or raises rates, your architecture is exposed.

Option B: Self-Hosted (vLLM / SGLang on Owned or Leased GPU Infrastructure)

Two A100 80GB GPUs on spot pricing cost approximately $1.80–2.50 per hour, translating to $1,300–$1,800 per month in raw compute. At 100 million tokens per month on that hardware, your amortized cost per token drops to roughly $0.013–$0.018 per million — 30 to 50 times cheaper than API rates at equivalent throughput. vLLM and SGLang both support prefix caching and speculative decoding, which extend that advantage further on workloads with repeated prompt structures.

DeepSeek R1's MoE architecture illustrates why model architecture matters to the self-hosting calculus. Sparse activation means only a fraction of the 671B parameters execute per forward pass. Self-hosting DeepSeek R1 on appropriate hardware has been cited at approximately $0.11 per million tokens versus $0.55 via API — a 5× spread that dramatically compresses the break-even threshold compared to a dense 70B model.

The costs that don't appear in the token math: 0.5–1 FTE of MLOps or DevOps time for initial deployment and ongoing maintenance, hardware procurement lead times of 4–8 weeks, fault-tolerant architecture to handle spot instance interruptions, and manual model update cycles. For a 405B-class model like Llama 3.1 405B, you're looking at eight H100s and $200,000+ in hardware before you serve a single production token.

Cost Comparison at Key Volume Thresholds

Monthly Token Volume	API Cost (70B, ~$0.75/M avg)	Self-Hosted Cost (2× A100 spot)	Self-Host Advantage
10M tokens	~$750	~$1,500 (fixed floor)	API wins
50M tokens	~$3,750	~$1,500–1,800	Near break-even
100M tokens	~$7,500	~$1,800–2,200	Self-host 3–4× cheaper
200M tokens	~$15,000	~$2,200–3,000	Self-host 5–6× cheaper

For MoE models like DeepSeek R1, the break-even threshold shifts downward significantly due to lower per-token compute requirements at inference time.

Decision Framework

Four questions determine the right answer for your team. Work through them in order — each can short-circuit the rest.

Token volume. Project your monthly token consumption 90 days forward, not today's usage. Above 100 million tokens per month, the cost case for self-hosting is unambiguous. Between 50 and 100 million, you're in hybrid territory. Below 50 million, the API default holds on pure economics.

Data sensitivity. Does your workload touch PII, PHI, financial records, or fall under EU AI Act or sector-specific compliance requirements? If yes, this stops being an economics question and becomes a compliance obligation. Self-hosting is the answer regardless of where your token volume sits.

Latency requirements. Does your application require sub-100ms inference — real-time voice, streaming agent loops, interactive coding assistants? The 100–400ms network overhead of an API call may be architecturally disqualifying at that latency target, independent of cost.

Operational capacity. Can your team staff 0.5–1 FTE of infrastructure expertise for GPU cluster management, vLLM configuration, and on-call incident response? If you're factoring in hiring costs to answer this question, factor those into your TCO before comparing token prices.

Mapping these across two dimensions — token volume on one axis, data sensitivity plus latency requirements on the other — produces four clear outcomes: API-only for low volume and low sensitivity, hybrid for moderate volume or mixed sensitivity requirements, self-hosted priority for high volume or latency-critical workloads, and self-hosted required when data residency is non-negotiable regardless of scale.

Our Recommendation

Default to API if you're under 50 million tokens per month and have no hard compliance constraints. Together AI and Groq both offer open-weight access with predictable pricing and no lock-in to a single proprietary vendor. The operational overhead of self-hosting at that scale erases any theoretical savings.

Self-host if you're above 100 million tokens per month or if your workload is subject to data residency requirements. At 200 million tokens per month, the monthly savings versus API pricing typically exceed the all-in DevOps overhead cost, delivering positive ROI within three to six months. Use vLLM or SGLang as your inference engine — both are production-grade, actively maintained, and support the optimization features (prefix caching, tensor parallelism, speculative decoding) that keep per-token costs low as models scale. For MoE architectures like DeepSeek R1, the economics tilt toward self-hosting at even lower volume thresholds, making it worth running the numbers specifically for that model class.

For teams in the 50–100 million token range, run a hybrid architecture: self-host your highest-volume, data-sensitive workloads and route lower-volume or burst traffic to API endpoints. Approximately 45% of Fortune 500 AI teams already operate this way (Gartner, Q4 2025). The infrastructure complexity is manageable, and it keeps your API spend from compounding as volume grows while avoiding full capital commitment before you've validated production traffic patterns.

Revisit this analysis every time a new model generation ships, every time your monthly token consumption crosses a major threshold, and every time hardware pricing shifts materially. With B100/GB200 availability continuing to reshape self-hosting TCO and API providers competing aggressively on open-weight pricing, the right answer today may not be the right answer in eight months.