#rate-limiting

#rate-limiting

In recent blog posts, both Uber ( Uber's Rate Limiting System), and OpenAI ( Beyond rate limits: scaling access to Codex and Sora) discuss shifts in their approach to rate limiting: moving from counter-based, per-service limits to adaptive, policy-based systems. Both companies developed proprietary rate-limiting platforms implemented at the infrastructure layer. These systems feature soft controls that manage traffic by asserting pressure on clients rather than utilizing hard stops - either through probabilistic shedding or credit-based waterfalls - ensuring system resilience without sacrificing user momentum.

GitHub engineers recently traced user reports of unexpected "Too Many Requests" errors to abuse-mitigation rules that had accidentally remained active long after the incidents that prompted them. According to GitHub, the affected users were not generating high-volume traffic; they were "making a handful of normal requests" that still tripped protections. The investigation found that older incident rules were based on traffic patterns that were strongly associated with abuse at the time, but later began matching some legitimate, logged-out requests.