The default and most commonly used table engine in ClickHouse, MergeTree, is optimized for high-throughput batch inserts. It writes each insert as a separate partition, then runs background merges to keep data manageable. This makes writes very fast, but not when they arrive in lots of tiny batches, which was exactly our case with millions of individual devices uploading one log event every 2 minutes.
Stephanie Cohen, Cloudflare's chief strategy officer, stated, 'The change in traffic patterns has been rapid, and something needed to change. This is just the beginning of a new model for the internet.'