Artificial intelligence
fromArmin Ronacher's Thoughts and Writings
6 days agoLLM APIs are a Synchronization Problem
APIs for large language models are an inadequate abstraction; the real problem is distributed state synchronization involving token histories and GPU KV caches.