The Economics of Tokens

For most companies, software spending has traditionally been relatively easy to categorize. There are licenses, infrastructure, development costs, support contracts and cloud services. Artificial intelligence introduces a less familiar component: the work performed by the software itself can now carry a direct, variable cost.

At small scale, this is easy to ignore. A few employees using an AI assistant, or an experimental application making occasional model calls, will rarely produce a bill large enough to influence a business decision. The economics begin to change when AI moves beyond chat and becomes embedded in applications and operational processes.

A customer request may trigger a model call. A quotation system may use AI to interpret requirements. A support application may summarize a case, retrieve documentation, compare it with internal rules and prepare a response. What appears to the user as one operation may actually involve several separate calls to one or more models, each consuming computational resources and adding to the cost of completing the task.

This is why AI planning cannot be reduced to choosing the most capable model. Architecture, usage, data handling, performance and cost have to be considered together.

From subscriptions to consumption

Chat products conceal much of this complexity. A company buys subscriptions for its employees and receives a reasonably predictable monthly expense. Users open the application, ask questions and get answers. The relationship between usage and cost exists, but it is largely abstracted away.

Applications work differently. When software connects directly to AI models through APIs, the company decides what information is sent, how much context accompanies each request, how frequently calls are made and whether additional calls are required to verify, transform or act on a response.

Consider a document-processing workflow. One model may classify an incoming document, another step may extract relevant information, a further call may compare the result with company rules, and a final model may compose a response or recommend an action. To the employee, this is one process. To the infrastructure, it may be four or five separate AI operations.

That difference matters because tokens, despite being useful for billing, are not a particularly useful business metric. Management does not normally think in millions of tokens. It thinks in quotations, support cases, documents, transactions and customers.

The meaningful question is therefore not simply how many tokens an application consumes, but what the AI component costs per completed unit of work. Cost per quotation, cost per support case, cost per document processed or cost per successful automated operation allows AI expenditure to be compared with the rest of the business.

This also exposes an important problem with apparently cheap models. Two systems may consume a similar number of tokens and still have very different economics. One may solve a task correctly on the first attempt; another may need retries, additional context or validation from a more capable model. The cheaper model can easily produce the more expensive process.

Application design has the same effect. Sending an entire customer history on every request is economically different from retrieving only the information relevant to the current operation. Reprocessing stable information repeatedly is different from caching or reusing it. AI cost is therefore partly a model-pricing question, but increasingly an engineering question.

Not every task needs the best model

Once AI becomes part of a production system, model selection itself becomes a form of cost control.

There is little economic justification for sending every operation to the most capable model available. A difficult contractual analysis may justify an advanced reasoning system; identifying the language of an incoming email probably does not. Classification, extraction, summarization, translation and structured-data generation can often be handled adequately by smaller models, while more difficult cases are escalated when necessary.

This leads naturally to a tiered architecture. Routine work can be assigned to lower-cost models, while expensive capability is reserved for cases in which it produces a measurable benefit. The distinction can even happen inside a single process: a smaller model performs the first classification, for example, and a stronger model is called only when the case falls outside normal parameters.

The objective is not to use cheaper AI everywhere. It is to stop paying premium intelligence prices for tasks that do not require premium intelligence.

The same logic applies to infrastructure. Companies now have several practical ways to run AI. They can consume proprietary models directly from providers, access commercial models through major cloud platforms, deploy open-weight models on managed infrastructure, or operate models on dedicated or local hardware.

Each option changes the economic equation. Direct access to a commercial provider is usually the simplest place to start. Infrastructure requirements are minimal, advanced capabilities are immediately available and costs scale with usage. In exchange, the company accepts a degree of dependence on the provider's pricing, availability, contractual conditions and technical environment.

Open-weight models offer a different balance. They can be deployed in the cloud, on dedicated servers or inside the organization, giving companies greater control over data and infrastructure while transferring more operational responsibility to them. Hardware, maintenance, security, updates, capacity planning and technical support become part of the equation.

There is no universally superior model. The correct choice depends on the workload.

This becomes particularly interesting after an AI process has been running for some time. During development, using a strong commercial model may be the fastest way to determine whether an idea works at all. Months later, the company has something far more valuable than speculation: operational data. It knows which tasks are repetitive, which genuinely require sophisticated reasoning, what traffic looks like and how much information is being processed.

At that point, parts of the workload may become candidates for smaller or open-weight models. A premium commercial system might continue handling complex exceptions while routine classification, extraction or internal knowledge tasks move elsewhere. This is not necessarily a philosophical choice between proprietary and open AI. It is workload allocation.

Cloud, local infrastructure and the utilization problem

Cloud services have an obvious economic advantage at the beginning: they require little capital investment and allow companies to pay largely according to usage. This makes them particularly attractive for variable workloads, pilots and applications whose future demand is still uncertain.

Dedicated infrastructure behaves differently. Hardware, electricity, redundancy, maintenance and technical support become fixed costs. Once that capacity exists, however, additional inference can have a relatively low marginal cost.

The deciding factor is utilization. Buying expensive computing hardware for a process that runs occasionally makes little sense. Keeping the same hardware busy with a large, predictable workload can produce very different economics.

Executives do not need to become specialists in GPU utilization, but they should expect the architecture team to understand this crossover. Before a major AI workload becomes permanently attached to one provider or one infrastructure model, there should be a credible comparison between consumption-based pricing and the cost of operating dedicated capacity.

Compliance belongs in that calculation as well. The cheapest technical architecture is not necessarily the cheapest business architecture. Customer records, financial information, legal documents or internal operational data may impose geographic, contractual or industry-specific requirements. Sending them to an external service can require additional controls and approval processes. Running models internally may alleviate some concerns while creating new responsibilities around access control, security, maintenance and model governance.

Those costs are real even if they never appear on a model provider's invoice.

AI needs financial observability

Companies already monitor production applications for availability, errors and performance. AI-enabled systems need an additional form of observability: consumption.

For every important AI process, someone should be able to determine which models are being used, how frequently they are called, what they cost and what business activity is generating that expense. The information does not need to be presented in technical language.

A management dashboard might show, for example, that customer-service automation processed 18,000 cases during the month at an average AI cost of fourteen cents per case. It might show that 82 percent were handled by a smaller model while 18 percent were escalated to a more capable one, and whether the average cost per case is rising or falling.

That tells management something useful. A provider invoice containing a large token count does not.

Scale also needs to be included before deployment. AI pilots are frequently inexpensive because very little is happening. Successful production systems are not. A feature that initially serves a small group may eventually run thousands or millions of times, while an internal assistant may become embedded in workflows across an entire organization.

The relevant questions are familiar capacity-planning questions: what does the process cost at today's volume, what happens at five or ten times that volume, at what point does another model become economically attractive, and when would dedicated infrastructure begin to make sense?

AI introduces a new variable, but the discipline required to manage it is not new.

Build around capabilities, not models

There is one architectural decision that can make all of this considerably easier: applications should remain reasonably independent from the individual models behind them.

The AI market is moving too quickly to assume that today's preferred model will remain the preferred model for the life of an enterprise application. Capabilities improve, prices move, providers emerge, open-weight systems become stronger and hardware becomes more efficient.

An application built tightly around one model can turn a routine commercial decision into a software redevelopment project.

A better architecture separates the business capability from the system that provides it. The application asks for something to be done: classify this document, extract these fields, analyze this case, generate this response. A routing or orchestration layer decides how that capability should be provided.

That layer can take into account price, latency, security, model capability or even current availability. A routine request might go to an inexpensive model. A difficult one might be escalated. Sensitive information might remain inside controlled infrastructure. A high-volume workload might migrate to dedicated hardware once its usage becomes predictable.

Model choice then becomes a routing decision rather than a permanent architectural commitment.

This is likely to become normal. Enterprise AI will probably grow more heterogeneous, not less. A company may simultaneously use several commercial providers, open-weight models and local infrastructure, with each serving the workloads for which it makes economic and operational sense.

The companies best prepared for that environment will know three things: how much their AI-enabled processes actually cost, which parts of those processes genuinely require expensive intelligence, and how easily those workloads can be moved when technology, pricing or regulation changes.

Tokens are only the current mechanism used to measure consumption. The larger management issue is that AI introduces a variable cost directly into business processes.

Once applications begin consuming intelligence continuously, that consumption stops being an experiment. It becomes a production resource, and it has to be designed, measured and optimized accordingly.