Every Obvious Approach Failed: Building an Async Database Client That Actually Scales (Sponsor: ClickHouse)

I maintain clickhouse-connect, the official ClickHouse Python client. Our first async implementation followed the standard playbook of wrapping the synchronous client in run_in_executor. That works until large result sets and real concurrency show up and tail latency starts spiking in ways ordinary profiling doesn’t explain. The problem isn’t specific to ClickHouse. When a database can stream data efficiently but decoding that data is CPU-heavy, network I/O and parsing stop being independent. You can make the socket non-blocking and still end up with an async system that behaves badly under load. This talk walks through the obvious designs I tried, why each one failed, and the architecture that finally held up which is async I/O on one side, synchronous CPU-bound parsing on the other, and a bounded queue between them to provide backpressure. I’ll show the pattern in the context of a database client, then generalize it to other Python systems that mix async I/O with heavy CPU work.