For the last few years, adopting AI inside a business has usually meant connecting to somebody else’s model.
A company builds an assistant, automation or internal tool, sends requests to an API from OpenAI, Anthropic or another provider, receives the response and pays for the usage.
That approach still makes sense. It gives businesses immediate access to extremely capable models without having to operate the underlying AI infrastructure themselves.
But it is no longer the only realistic option.
Open-weight models have improved considerably, the hardware requirements for useful local inference have fallen, and businesses can now run surprisingly capable AI models on infrastructure they control.
That creates a question we think more companies should be asking:
Does every employee, workflow and AI request actually need access to the most powerful model available?
For a large proportion of everyday business AI, the answer is probably no.
And if the task doesn’t require a frontier model, repeatedly sending it to one through an external API may no longer be the best architecture.
The best AI model isn’t necessarily the best model for the job
AI companies have good reasons to keep building more capable models. Better reasoning, coding, multimodal understanding and agent capabilities allow AI to solve increasingly difficult problems.
But improvements at the frontier do not mean every business task suddenly requires frontier-level intelligence.
There is a substantial difference between asking an AI model to solve a difficult software engineering problem and asking it to categorise an incoming support request.
The same applies to summarising a meeting, extracting information from a document, rewriting an email or looking up an internal policy.
These are useful applications of AI, but they are not necessarily difficult AI problems.
This distinction matters because businesses often approach AI model selection backwards. They begin by asking:
“What is the best AI model available?”
A more useful question is:
“What is the smallest and most efficient model that can perform this particular task reliably?”
It is similar to the principle we discussed in What Should You Automate Before Building Custom Software?: technology should be selected around the problem being solved, rather than deploying the most complex option simply because it is available.
Open models are getting good enough to change the decision
This argument becomes much more important as open-weight models improve.
In its 2026 AI Index, Stanford reported that, as of March 2026, the performance gap between the leading closed and open models on its tracked comparison was 3.3%.
That does not mean an open model is automatically equivalent to the best proprietary model for every task. Stanford also highlights problems with AI benchmark reliability, and benchmark performance should never be confused with guaranteed performance inside a real business workflow.
But it does demonstrate something important: useful AI capability is no longer confined to a handful of closed models available only through external APIs.
OpenAI itself is now part of this shift.
Its gpt-oss open-weight models were released specifically so they can be downloaded and run outside the OpenAI API. OpenAI says the smaller gpt-oss-20b model can run on systems with as little as 16GB of memory and describes it as suitable for local inference and specialised use cases.
The larger gpt-oss-120b model can run within 80GB of memory on a single suitable GPU.
Those requirements still represent real computing infrastructure, particularly for the larger model. But they are a long way from the assumption that capable language models can only be operated inside the data centres of the largest AI companies.
We saw an earlier version of this shift when DeepSeek R1 challenged assumptions about the cost and concentration of advanced AI development. The wider direction has continued: businesses now have considerably more choice about where their models run and who controls the infrastructure.
Most employees aren’t solving frontier AI problems
Look at what AI is likely to do during an ordinary day inside a company.
| Typical business AI task | Does it normally require a frontier model? |
|---|---|
| Summarising an internal document | Usually not |
| Categorising incoming enquiries | Usually not |
| Extracting names, dates or reference numbers | Usually not |
| Rewriting an email | Usually not |
| Searching company documentation | Usually not |
| Turning meeting notes into actions | Usually not |
| Drafting routine internal content | Usually not |
| Complex software engineering | Possibly |
| Difficult multi-stage reasoning | Possibly |
| Highly specialised analysis | Possibly |
The exact answer depends on the task, the required accuracy and the model being evaluated. A smaller model should never be assumed to be good enough without testing it against the actual workload.
But that testing is precisely the point.
If an employee needs AI to extract an order number from an email, the fact that another model is substantially better at advanced mathematics may have no commercial value whatsoever.
If an internal assistant needs to answer questions from a controlled set of company documentation, the ability to discuss thousands of unrelated subjects may not be particularly useful either.
A company may therefore be paying for model capabilities that a large proportion of its AI requests never need.
Local AI changes the economics of high-volume usage
Using an API has a major advantage: somebody else operates the infrastructure.
You do not need to provision GPUs, deploy the model, manage updates or monitor inference infrastructure. You make a request and pay for the service.
For experimentation, low usage or workloads that change dramatically from one request to another, that can be an excellent arrangement.
But the economics become more interesting once AI becomes part of normal business infrastructure.
Imagine dozens or hundreds of employees using AI throughout the working day.
Then add automated systems processing documents, categorising emails, analysing enquiries, searching internal information and generating routine outputs without a person manually starting each request.
Every one of those interactions can become another billable API inference.
A self-hosted model changes that relationship. The business pays for the infrastructure needed to operate the model rather than purchasing each request from the model provider.
That does not mean local AI is automatically cheaper.
OpenAI itself makes this distinction in its guidance for gpt-oss deployments. It notes that self-hosting may be cheaper in some circumstances, while its API can remain more economical once the costs of hosting, maintenance and upgrades are taken into account.
That is exactly how businesses should evaluate the decision.
A GPU sitting mostly idle is not a cost saving. Neither is a local deployment that requires disproportionate engineering effort to maintain.
But if a company has a large, predictable volume of AI requests that can be handled reliably by a smaller model, the economics can move strongly in favour of operating dedicated inference capacity.
Smaller models can also mean less inference overhead
A smaller model does not simply mean a worse version of a larger one.
Model architecture, quantisation, fine-tuning and inference optimisation can significantly change the resources required to run AI.
Nort has previously covered some of these techniques in our article on AI model optimisation and more efficient inference.
The practical business implication is straightforward: if a task can be performed reliably by a smaller model, there may be little benefit in allocating substantially more computing power to it.
This matters particularly for automated workloads.
A staff member manually asking a few questions each day may not produce meaningful inference costs. A background process analysing thousands of pieces of text can.
Once AI becomes embedded into software rather than simply used as a chatbot, efficiency starts to matter much more.
Keeping AI local also changes the data relationship
Cost is only part of the argument for local AI.
Companies increasingly want AI to work with information that is not public: contracts, internal procedures, customer records, technical documentation, product information, financial data, support conversations and intellectual property.
It is important not to misrepresent how the major commercial AI providers handle this data.
OpenAI states that it does not use inputs or outputs from its API and business products to train its models by default. Anthropic similarly states that inputs and outputs from its commercial products, including the Anthropic API, are not used for model training by default.
So the case for local AI should not be based on pretending that sending a request to an enterprise API automatically means your confidential information becomes training data.
It doesn’t.
The stronger argument is about control of the processing environment.
With a genuinely self-hosted model, the organisation can choose to keep inference inside its own infrastructure or private environment rather than sending the content to an external AI provider at all.
OpenAI says that when its gpt-oss models are operated on infrastructure controlled by the user, OpenAI does not receive or process the information sent to those models unless the user explicitly shares it with OpenAI or uses a managed hosting partner.
For organisations dealing with commercially sensitive information, regulated data or valuable intellectual property, that architectural difference can matter even when the external provider already offers strong contractual privacy protections.
Local does not automatically mean secure
There is an important caveat.
Running a model yourself does not magically make an AI system secure.
The organisation becomes responsible for the infrastructure around it: authentication, permissions, network security, logging, data access, software updates, model access and any tools or databases connected to the AI system.
A badly designed local deployment can be less secure than a professionally managed enterprise API.
The advantage is not automatic security.
The advantage is that the organisation has greater control over the architecture and can decide where processing occurs, which systems the model can access and what information leaves its environment.
Your company probably needs company-specific AI more than world-class general knowledge
There is another reason smaller local models can make sense.
Businesses often assume the useful part of an AI system is how much general knowledge the underlying model contains.
For internal AI, that is often only part of the equation.
An HR assistant needs access to the company’s actual HR policies.
A product-support assistant needs current product documentation.
An operations assistant needs access to the relevant systems and processes.
A document-processing model needs to extract the correct information consistently.
In these situations, an AI system that understands the organisation’s specific environment can be more useful than a more powerful general model with no access to the information needed to complete the job.
That company-specific capability can be built in several ways. It may involve retrieval-augmented generation, controlled context, fine-tuning, tool access or integration with internal systems.
It usually does not mean training a foundation model from scratch.
Open-weight models make this architecture more attractive because they can be adapted and fine-tuned without the organisation being tied exclusively to the model provider’s hosted inference service.
Frontier APIs still have a place
Moving more AI workloads locally does not mean OpenAI, Anthropic and other frontier providers suddenly become unnecessary.
There are tasks where using the strongest available external model is entirely rational.
Difficult reasoning, advanced coding, unfamiliar problems, certain multimodal workloads or tasks where a particular frontier model has demonstrated materially better performance may justify the additional cost and external processing.
APIs are also extremely useful when a company is testing an idea. Building infrastructure before establishing whether an AI workflow is useful would often be backwards.
The important change is that a frontier API no longer needs to be the automatic destination for every request.
The stronger architecture may be local first, frontier when necessary
This is where we think the business case becomes particularly interesting.
The choice does not need to be:
Local AI or OpenAI.
Nor does it need to be:
Local AI or Anthropic.
A company can use different models for different levels of work.
Routine document processing can happen locally.
Internal company Q&A can run against a locally controlled model and knowledge base.
High-volume classification and extraction workloads can use a smaller model selected specifically for those tasks.
Sensitive information can be restricted to models running within the company’s chosen environment.
Then, when a request genuinely requires greater reasoning capability, the application can route that request to a frontier model.
The frontier model becomes an escalation path rather than the default destination for every inference.
That is closer to how good software architecture already works. We do not normally allocate the maximum possible computing resources to every operation simply because those resources exist.
We match infrastructure to workload.
AI should be treated the same way.
Businesses should audit their AI workloads before their API bills become infrastructure
The continuing improvement of frontier AI models is good news for businesses.
But the equally important development is that useful AI is becoming available at more model sizes, with more deployment options and far more organisational control.
That means businesses should stop treating AI model selection as a leaderboard competition.
Instead, look at the actual workload.
What are employees asking the model to do?
Which requests contain sensitive information?
Which tasks happen thousands of times?
Which genuinely need advanced reasoning?
Which could be performed by a smaller model?
And which workloads would make more sense running on infrastructure the business controls?
For organisations using AI extensively, the answer is unlikely to be one model for everything.
Our view at Nort Labs is that considerably more routine business AI can move towards locally controlled models, while frontier APIs remain available for the smaller set of tasks where their additional capability genuinely earns its cost.
That can mean lower inference costs at scale, greater control over private company information and AI systems designed around the organisation’s actual work rather than around whichever model currently sits at the top of a benchmark.
If your organisation is already making substantial use of OpenAI, Anthropic or another AI API, Nort Labs can assess which workloads are suitable for local AI, which should remain on frontier models and how a hybrid architecture could be designed around your existing systems.
