Tokens are the fundamental units of data processed by large language models to interpret meaning in sentences. Tokens can represent whole words, sub-words, or strings of letters, symbols, or phrases, and compound words may split into multiple tokens. Token counts can be estimated at about three-quarters of a word per token, so 100 words can equal roughly 135 tokens. Token volume is used to measure AI growth, with Google processing 3.2 quadrillion tokens per month. Tokens also function as a metering and pricing mechanism for AI vendors, where demand drives up GPU strain and operating costs. Token pricing can differ depending on whether tokens are uploaded or downloaded.
"Google has only one way to measure the phenomenal AI growth it's seen: in tokens. The company processes 3.2 quadrillion tokens per month, Google CEO Sundar Pichai said during this week's I/O keynote, adding, "never imagined I'd say quadrillion..., but here we are.""
"Basically, tokens are a unit of measure used by large language models (LLMs) to process data. Tokens, which have been called the "new oil" fueling the AI revolution, are also a way AI vendors can meter usage and price their services. Enterprises are lusting for tokens, and spending billions of them to grab compute time."
"Similar to the way humans think, LLMs grasp the meaning of a sentence by breaking words down into tokens. Pichai described them as "the fundamental units of data our models process, many representing a problem being solved." The fundamental unit could be in the form of a word, a sub-word, or a string of letters, symbols, or phrases. Compound words can be split into multiple tokens."
"Not all tokens are priced the same. An uploaded token to an AI system is cheaper, while downloaded tokens are more expensive. A user, for instance, might pay to upload a resume, then pay even more to download the resume polished by an LLM."
Read at Computerworld
Unable to calculate read time
Collection
[
|
...
]