Cloudflare chose TimescaleDB over ClickHouse for its analytics and reporting needs focusing on a balance of simplicity and performance. The team developed Digital Experience Monitoring (DEX) to observe device and application performance across Zero Trust environments. They implemented a configuration plane for managing tests and an analytics plane for data ingestion and visualization. PostgreSQL was used initially for its capability to handle high-frequency event logging, which was necessary due to the large volume of data from individual devices. However, PostgreSQL alone could not meet the scaling demands as adoption increased.
The default and most commonly used table engine in ClickHouse, MergeTree, is optimized for high-throughput batch inserts. It writes each insert as a separate partition, then runs background merges to keep data manageable. This makes writes very fast, but not when they arrive in lots of tiny batches, which was exactly our case with millions of individual devices uploading one log event every 2 minutes.
To keep the initial release simple and deliver a working DEX MVP within four months, the team used PostgreSQL for both configuration data and analytical logs, handling 200 inserts per second at launch and query latencies in the hundreds of milliseconds for most customers.
Collection
[
|
...
]