Self-hosted AI can be considerably cheaper than paying an external model provider for every request. But that does not mean buying a server automatically saves money.
For a company making a few thousand AI requests a month, an API from OpenAI, Anthropic or another provider may be difficult to beat. There is no AI infrastructure to operate, capacity can increase instantly and the business only pays when it actually uses the model.
The calculation changes when AI becomes part of normal business infrastructure.
If employees, internal assistants and automated systems are continuously sending predictable work to external models, the company is effectively renting AI inference every time a task runs.
At sufficient volume, owning the capacity to perform much of that work starts to look very different.
This follows directly from the question we raised in Does Every Employee Really Need the Most Powerful AI Model?. If a large proportion of ordinary company AI work does not require a frontier model, the next question is obvious:
Why keep paying frontier API prices for work a smaller model could perform locally?
API-based AI is cheap to start because somebody else owns the infrastructure
The appeal of an AI API is easy to understand.
A developer does not need to buy GPUs, deploy a model, manage inference software or plan capacity. The application sends tokens to the provider and receives tokens back.
The provider handles the underlying infrastructure.
That turns what could be an infrastructure project into a variable operating cost.
As of September 2026, for example, OpenAI lists GPT-5.6 Sol at $4 per million input tokens and $20 per million output tokens. Anthropic prices Claude Sonnet 5 at $2 per million input tokens and $10 per million output tokens.
There are much cheaper hosted models too. OpenAI’s GPT-5.6 Luna, which is aimed at cost-sensitive, high-volume workloads, currently costs $0.20 per million input tokens and $1.20 per million output tokens.
| Model | Input per 1M tokens | Output per 1M tokens |
|---|---|---|
| OpenAI GPT-5.6 Sol | $4.00 | $20.00 |
| Anthropic Claude Sonnet 5 | $2.00 | $10.00 |
| OpenAI GPT-5.6 Luna | $0.20 | $1.20 |
Prices above are provider list prices checked in September 2026. AI pricing changes frequently, and these models are not equivalent in capability.
Those numbers also demonstrate why simply saying “local AI is cheaper” is too simplistic.
If a lightweight hosted model performs the task perfectly well and the company barely uses it, running dedicated infrastructure could cost more.
But token pricing has another characteristic that matters as companies scale: every additional piece of work creates additional usage.
What self-hosted AI changes about the cost model
Self-hosted AI works differently.
Instead of buying inference from a provider request by request, the organisation runs a model on infrastructure it controls, whether that is an on-premises machine, its own servers or a private cloud environment.
Some model weights themselves can now be used without a per-token licence charge. OpenAI, for example, says its gpt-oss open-weight models are free to download and use under the Apache 2.0 licence and its usage policy. They are not served through the OpenAI API, so OpenAI API pricing and rate limits do not apply.
That does not make inference free.
The business still has to pay for the resources that perform it:
- GPU or other computing hardware;
- electricity or cloud compute;
- storage;
- inference software;
- monitoring and maintenance;
- engineering time;
- redundancy and additional capacity where required.
The important difference is what the company is paying for.
| Hosted API | Self-hosted model |
|---|---|
| Primarily variable usage cost | Primarily infrastructure and operating cost |
| More requests generally mean more token charges | Unused available capacity can process more work without an API charge for every token |
| Provider owns and manages model infrastructure | Organisation controls the deployment infrastructure |
| Very little upfront infrastructure required | Infrastructure has to be provisioned |
| Easy to absorb sudden demand | Capacity must be designed for expected demand |
| Model upgrades handled by provider | Model and inference stack require management |
That difference is what creates the potential saving.
The most important self-hosted AI number is utilisation
A local model sitting unused is not cheap AI.
If a business purchases expensive GPU capacity and uses it for ten minutes a day, the economics will probably look terrible compared with an API.
But imagine the same infrastructure processing requests throughout the working day: document extraction, internal search, customer-service classification, meeting summaries, content processing and automated workflows.
The hardware cost has not increased every time another request is processed.
That means the effective cost of each inference can fall as utilisation rises.
This is the central economic difference between the two approaches.
At a simplified level, API expenditure behaves something like this:
Monthly API cost = input usage + output usage + any tool or service charges
A self-hosted deployment behaves more like:
Monthly local cost = infrastructure + energy/hosting + maintenance + engineering + redundancy
The first increases closely with usage. The second has a much larger fixed-capacity component.
There will therefore be a point at which additional utilisation makes owned capacity more attractive. Where that point sits depends entirely on the workload and infrastructure.
High-volume repetitive work is where self-hosted AI becomes particularly attractive
The strongest financial case is not necessarily employees occasionally chatting with an AI assistant.
It is AI embedded into business software.
Consider an application that processes every customer email. Or a document system that analyses every file uploaded to the company. Or an internal knowledge assistant being used across hundreds of employees.
These systems can generate far more inference than somebody manually opening a chatbot several times a day.
The workload can also be much more predictable.
A document-classification system may repeatedly receive similar input and need a tightly structured output. A support-routing system may perform the same classification task thousands of times. An internal assistant may answer questions from the same controlled knowledge base.
These are exactly the types of tasks where asking whether a smaller or more specialised model can do the job becomes commercially important.
As we discussed in What Should You Automate Before Building Custom Software?, repetitive and rules-driven workloads are often where automation has the clearest value. Once AI becomes part of those automated processes, inference cost stops being an abstract technical metric and becomes an operating cost.
Long prompts can make the API calculation change surprisingly quickly
Counting requests alone can also hide the true cost of an AI application.
APIs are generally charged according to tokens, not simply how many times an application calls the model.
A thousand requests containing a short sentence are very different from a thousand requests containing lengthy documents, conversation histories or retrieved company information.
This matters particularly for business AI because company-specific systems often supply substantial context to a model.
An internal assistant might retrieve several relevant documents before answering a question. A contract-processing workflow might analyse pages of text. An agent may maintain a growing conversation history while working through a task.
The application can therefore consume large amounts of input even when the final response is short.
Hosted providers have developed ways to reduce this cost. Prompt caching, cheaper model tiers and batch processing can all materially improve API economics. Anthropic, for example, advertises substantial savings from prompt caching and 50% savings from batch processing for Claude Sonnet 5.
Those optimisations should absolutely be considered before declaring an API architecture expensive.
But they do not change the underlying distinction: the organisation is still purchasing inference from an external service rather than operating its own capacity.
Do not compare a local model only against the most expensive API
There is an easy way to make self-hosted AI look artificially cheap: compare a small local model against the most expensive frontier model available.
That is not a useful comparison.
If a low-cost hosted model can perform the job reliably, that should be part of the calculation too.
GPT-5.6 Luna’s current pricing demonstrates just how cheap hosted inference can become for suitable workloads. Likewise, model providers continue to reduce prices and introduce new cost-efficient tiers.
The correct comparison is:
What is the cheapest architecture that meets the required level of quality, reliability, speed, privacy and control?
Sometimes that will be an API.
But if a local model passes the same real-world evaluation, the business can then compare a recurring metered service with infrastructure it controls.
That is a much more interesting commercial decision.
A smaller local model can change the hardware requirement too
Businesses should also avoid assuming that local AI means operating the sort of infrastructure used to train a frontier model.
Training and inference are very different workloads.
OpenAI says its gpt-oss-20b model can operate with around 16GB of memory, while its larger 120b model can fit on a single 80GB GPU. Different open-weight models have different requirements, and production deployments may require additional capacity for concurrency and acceptable response speeds, but capable inference is increasingly possible on much smaller infrastructure than many organisations assume.
Techniques such as quantisation and model optimisation can reduce resource requirements further. We covered some of the underlying approaches in AI Model Optimization: Techniques for Faster and Efficient Inference.
This is another reason model selection matters financially.
If a company can reliably perform a task with a much smaller model, it may reduce both the cost of external inference and the infrastructure required to bring that inference in-house.
When is self-hosted AI likely to be cheaper?
There is no universal token count at which a company should switch. Hardware, models, concurrency, latency requirements and engineering costs differ too much for that.
But the economics generally become more favourable when several conditions exist together.
| More favourable for self-hosting | More favourable for an API |
|---|---|
| Large, sustained AI workload | Low or occasional usage |
| Predictable demand | Large unpredictable traffic spikes |
| Routine or specialised tasks | Work regularly requires frontier capability |
| A smaller model performs reliably | Only a large proprietary model meets the requirement |
| Infrastructure can remain well utilised | Dedicated hardware would sit idle |
| Company already has suitable technical resources | No desire to manage AI infrastructure |
| Data control has independent business value | External processing is acceptable |
The more boxes a business ticks on the left, the more seriously it should investigate local inference.
There is also a cost to remaining dependent on a provider
The monthly API bill is not the only economic factor.
An application built entirely around one external model provider can also inherit that provider’s pricing changes, rate limits, model retirements and product decisions.
That does not mean those providers are unreliable. Managed APIs remove a substantial amount of operational work, and model providers regularly make their services cheaper and more capable.
But dependency still has value and cost.
Open-weight models give businesses another option: the model can be operated, adapted and migrated within infrastructure chosen by the organisation.
For companies building AI deeply into their internal operations, that control can matter independently of the raw cost per token.
The cheapest architecture may still use both
There is no requirement to migrate every AI workload away from OpenAI or Anthropic.
In fact, doing so may be economically irrational.
A better architecture can be to process predictable high-volume work locally while retaining access to frontier APIs for requests that genuinely require them.
Routine document processing, classification, internal retrieval and repetitive workflow tasks can be candidates for self-hosted AI.
Unusually difficult requests can then be escalated to a more capable external model.
This prevents a company from paying frontier inference costs by default without giving up access to frontier intelligence when it provides genuine value.
It also means local AI does not have to replace an existing AI stack overnight. Workloads can be assessed and moved individually.
Before switching, calculate cost per successful task rather than cost per token
There is one final number that matters more than either API pricing or GPU pricing.
Did the model complete the job correctly?
A local model that costs half as much but requires twice as much human correction has not produced a saving.
Equally, paying for an advanced frontier model to perform a task that a much smaller model completes just as reliably is unnecessary expense.
Businesses considering self-hosted AI should therefore evaluate real examples from their own workloads rather than relying only on benchmark scores.
Measure accuracy, failure rates, latency, throughput and the amount of human intervention required. Then compare the total cost of producing an acceptable result.
That turns the decision from an argument about which AI model is “best” into a normal infrastructure decision.
Should your business move from AI APIs to self-hosted AI?
For light and unpredictable AI usage, probably not purely to save money. Hosted APIs are extremely convenient, and cheaper model tiers have made them increasingly competitive.
For companies running substantial, repeatable AI workloads, the answer can be very different.
Once a business is continually paying to process work that a smaller model could perform reliably, it is worth asking why that inference still needs to be rented one request at a time.
Self-hosted AI turns the calculation towards owned capacity. That brings infrastructure and maintenance responsibilities, but it can also reduce recurring inference costs, remove per-token pricing from suitable workloads and give the organisation much greater control over how its AI is operated.
Our view at Nort Labs is not that businesses should abandon OpenAI or Anthropic. It is that companies using AI at meaningful scale should stop assuming an external API is automatically the cheapest place for every workload to run.
If your organisation already has significant AI API usage, Nort Labs can help assess which workloads are suitable for local deployment, compare the economics with your existing API costs and build a self-hosted or hybrid AI system around the tasks your business actually performs.
