WTF are tokens?

"When someone sends a prompt to a large language model or receives a response, the system breaks language into small segments - pieces of works, full words, punctuation or even spaces. These fragments are called tokens. During training, the model learns how tokens fit together and which combinations tend to appear in sequence. When it generates a response, it predicts the next token, then the next, over and over."

"That's the technical explanation. The commercial one is simpler: tokens are how Ai usage gets metered and billed. Both the text you send in and the next the model generates are counted. The more tokens processed, the more compute is used - and the higher the cost. Why do tokens suddenly matter to agencies? Because they're using them more - and more consistently."

"Early AI use in agencies was sporadic - a prompt here, a deck summary there. Now AI is moving into always-on workflows: planning, reporting, segmentation, optimization and creative iteration. That changes usage. Instead of occasional prompts, agencies are running systems that monitor performance, forecast outcomes and assist decisions continuously. Each of those actions consumes tokens."

Tokens are the fragments of text—words, punctuation, spaces—that language models use during training and generation. Tokens measure both input and output and determine compute consumption. Higher token volumes require more compute and increase costs. Agencies are shifting AI from sporadic prompts to always-on workflows across planning, reporting, segmentation, optimization and creative work. Continuous monitoring, forecasting and decision-assist actions consume tokens and create persistent variable costs. Compute expenses therefore need forecasting, management and allocation within agency financial planning similarly to labor. Token metering is becoming a material operational and financial factor in knowledge work.

Read at Digiday