OpenAI Needs More Than Open Models: The Infrastructure

Updated 5 November 2026 · Standard formula, shown below

Yes: open models alone are not enough to make AI broadly useful. OpenAI—or any organization releasing open-weight models—also needs dependable infrastructure for training, serving, evaluating, securing, and updating them, while users need hardware, software, and expertise to run them well.

What “open” means for an AI model

The word “open” is used in several ways, and those meanings should not be confused. A model may have publicly downloadable weights without releasing its training data, complete training code, or every detail of its development. An open-source license may also impose different conditions from a license that simply permits people to use or modify model weights. Readers should check the specific release and its terms rather than infer openness from a label.

OpenAI’s name does not mean that all of its current models are open. The organization has released open-weight models, including the gpt-oss family announced in 2025, alongside products and models accessed through hosted services. Those are different ways to distribute AI: downloading weights gives users more control over where a model runs, while a hosted service handles much of the computing and maintenance for them.

The infrastructure behind a useful model

A model is a set of learned parameters, not a complete service. To turn it into something people can use, developers need a surrounding system that accepts requests, prepares inputs, runs computations, returns results, and monitors failures. Depending on the application, that system may include:

Compute: GPUs or other accelerators, memory, storage, and networking suitable for training or inference.
Serving software: tools that load model weights, manage requests, batch work, and make use of available hardware.
Application components: interfaces, databases, retrieval systems, and connections to other software.
Operations: logging, monitoring, version control, updates, capacity planning, and incident response.
Evaluation and safeguards: tests for quality, reliability, privacy, and misuse risks in the intended setting.

These needs vary substantially. Running a model for one person is different from serving thousands of users at once. A research experiment can tolerate occasional interruptions that a business-critical service cannot. Infrastructure is therefore not one fixed bundle; it is the set of resources and systems needed for a particular workload.

Training and inference are different jobs

Training is the process of adjusting model parameters using data and substantial computing resources. It requires a carefully managed pipeline for preparing data, distributing computation, saving checkpoints, and checking whether training is progressing as intended. Reproducing a published result can also be difficult if the dataset, code, hardware, or training details are unavailable.

Inference is the work of generating an answer from a trained model. It can be run on a personal computer, a server, or a cloud platform, but the practical requirements depend on model size, numerical precision, context length, and demand. A model that loads successfully may still respond slowly or exceed available memory when given long inputs or several simultaneous requests.

For example, a developer testing a model locally might use a compatible inference engine and a machine with sufficient memory. A public application may instead need multiple accelerators, request queues, traffic controls, and a plan for outages. The downloadable model is common to both cases; the service infrastructure is not.

What open weights give users—and what they do not

Open weights can offer meaningful advantages. Organizations may be able to run a model in their own environment, adapt it for a task, inspect its behavior, or avoid sending prompts to an external provider. Researchers can compare models more directly, and developers can build applications without relying on a single hosted endpoint.

But access to weights does not automatically provide:

the compute needed to run the model at a useful speed or scale;
the data and complete recipe needed to reproduce its training;
an application, support team, uptime commitment, or automatic updates;
proof that the model is accurate, unbiased, or safe for a specific use; or
permission to use it in every context—license terms still matter.

Another common misconception is that an open model is necessarily cheaper. It may reduce or remove per-request fees to a hosted provider, but hardware, electricity, engineering time, maintenance, and operational risk remain. Whether self-hosting saves money depends on the workload and the organization’s ability to run the system efficiently.

Choosing between local, self-hosted, and hosted use

The right setup follows from the requirements, not from the label “open.” A local model can suit experimentation or work where keeping data on a controlled device matters, provided the device can run it adequately. Self-hosting on organizational infrastructure offers more control over deployment, but the organization takes responsibility for availability, updates, and security. A hosted model service is often simpler to start with because the provider manages much of the compute and serving layer, though users have less control over the underlying deployment.

Before choosing, consider these questions:

How much traffic is expected, and how quickly must answers arrive?
What hardware and technical staff are already available?
Must prompts and outputs remain within a particular environment?
Does the model’s license permit the intended use and redistribution?
Who will test updates, investigate errors, and respond to outages?

For many teams, a small trial is more informative than a theoretical comparison. Test representative prompts, measure latency and resource use, and review errors before committing to a large deployment.

Safety, evaluation, and ongoing maintenance

Safety does not arrive automatically with model weights. The model may produce incorrect or misleading answers, reveal sensitive information included in its inputs, or be used in ways the developer did not anticipate. Applications can reduce some risks with access controls, careful data handling, limits on tool use, human review for consequential decisions, and clear user expectations. None of these measures guarantees safe behavior.

Evaluation should reflect the actual task. A general benchmark score cannot establish that a model is dependable for a particular company’s documents, languages, or users. Teams should test realistic examples, include difficult and adversarial cases, record known limitations, and repeat checks after changing the model, prompt, retrieval data, or serving software.

Maintenance matters because a working deployment can become unreliable as demand, software dependencies, or user needs change. Teams need a way to identify the model version behind an output, roll back a problematic update, monitor resource consumption, and handle failures. These operational practices are part of making an open model usable, not optional extras.

Frequently Asked Questions

Does OpenAI make all of its models open source?
No. OpenAI offers different kinds of models and access methods. Some releases provide downloadable open weights, while other models are available through hosted products. The release terms and included materials determine what is actually open.

Can I run an open-weight model without internet access?
Often, once you have obtained the weights and required software, inference can run locally without an internet connection. Initial downloads, updates, or applications that rely on online services may still need connectivity.

Do open models need powerful computers?
Not always, but hardware requirements depend on the model and workload. Smaller or quantized versions may run on more modest systems, usually with trade-offs in performance or output quality. Check the model’s documented requirements and test your intended use.

Are open models automatically private?
No. Local or self-hosted inference can reduce the need to send prompts to an outside service, but privacy also depends on logging, access controls, storage, telemetry, and the application’s data practices.

Is self-hosting always cheaper than using an API?
No. Self-hosting shifts costs rather than eliminating them: hardware or cloud compute, electricity, engineering, maintenance, and downtime all matter. Compare the total cost for your expected usage, not just the price of a model download.

Method: the standard formula for this calculation, rounded as each result states. See how the calculators work or report an error.