Artificial intelligence
fromArmin Ronacher's Thoughts and Writings
1 day agoLLM APIs are a Synchronization Problem
APIs for large language models are an inadequate abstraction; the real problem is distributed state synchronization involving token histories and GPU KV caches.