The duality of language models in the browser

"AI in the browser reeks of a product manager trying to hit a KPI to shoehorn AI into everything. I understand if you read the phrase “AI in the browser” and involuntarily threw up on your keyboard, said “Fuck this”, and then closed this tab. I will not hold that against you. For the rest of you, hear me out..."

"There's an “Always bet on the Web” quality to the idea of browser-based models that I appreciate. There's a self-hosted vibe to it. I shouldn't have to pay a billionaire each month to summarize an email or generate a picture of myself with ultra-white teeth, doubly-so when they're selling our regurgitated data back to us. Those gimmicky use-cases or even the more practical-yet-often- purple-washed “AI for accessibility” use-cases should fall under the Web's umbrella of universal access."

"I like that nobody expects much from them. To the hyper-scalers SLMs will always be the shittier version of LLMs so they're basically ignored. They dodge the whole “model get better” hype-chamber because they don't economically depend on the hopium that response quality will get better in the next version. Ironically, there was a breakthrough in small language models last month, but it barely got a mention on The Verge."

"Small language models generally... Cost less energy to train and run Are local, offline, and private Have a higher probability of ethically-sourced content Are free to use and don't charge per-token Tend to be open-source/weights. Don't require a backend server or API keys."

Small language models in Chrome and Edge are available behind experimental flags. Browser-based AI can provide universal access without paying large providers for basic tasks like summarizing emails or generating images. These models are often treated as lower quality than larger models, which reduces hype and avoids reliance on continual quality-improvement promises. Recent progress in small language models has received limited attention. Small models can generate examples of capability without large data-center compute. They cost less energy to train and run, can run locally offline, and support privacy. They are more likely to use ethically sourced content, are free to use without per-token charges, often open-source with available weights, and do not require backend servers or API keys.

Read at daverupert.com