"DoorDash has built and deployed an AI-driven safety system called SafeChat to moderate conversations between Dashers and customers across in-app chat, images, and voice calls. Safechat applies machine learning to detect and respond to unsafe content in near real time, screening communications for offensive or inappropriate material and enabling immediate actions such as reporting issues or unassigning deliveries. SafeChat focuses on safety rather than engagement or automation, positioning AI as core infrastructure to protect platform integrity and the well-being of Dashers and customers."
"DoorDash engineers first used a three-layered approach in Phase 1. The first layer, a moderation API, provided a low-cost, high-recall filter that automatically cleared about 90 percent of messages with minimal latency. Messages not cleared proceeded to a fast, low-cost large language model (LLM) with higher precision, identifying 99.8 percent of messages as safe. The remaining messages were evaluated by a more precise, higher-cost LLM, scoring messages across profanity, threats, and sexual content."
SafeChat is an AI-driven safety system that moderates Dashers' and customers' in-app chats, images, and voice calls to detect unsafe content and enable immediate safety actions. The system uses a layered architecture combining machine learning models and human review for escalation, handling millions of interactions daily. Phase 1 applied a three-layer pipeline: a low-cost moderation API clearing about 90% of messages, a fast LLM identifying 99.8% of remaining messages as safe, and a precise higher-cost LLM scoring profanity, threats, and sexual content to trigger actions like order unassignment. Phase 2 introduced an internal model trained on roughly 10 million messages to improve scalability, reduce costs, and sustain 99.8% traffic handling with low latency.
Read at InfoQ
Unable to calculate read time
Collection
[
|
...
]