Apache Kafka's ghost messages: aborted transactions and read_uncommitted
Why some Kafka clients show you messages that, for the producer, never existed, and how to spot them. Isolation level, LSO and transaction markers explained with an example.
Blog
The Topicup blog
Apache Kafka in practice: real problems, solutions, challenges and curiosities, with numbers and the conditions we measured them in.
Searching for a value across millions of Kafka messages: how long it really takes
Apache Kafka has no index on message content: to find a customer code inside a JSON you have to read. How to estimate the time, why bytes matter more than messages, and three ways to read less.
The regex that never finishes: backtracking and large Kafka messages
A harmless-looking regular expression can stall a search for hours on a single 1 MB message. Why it happens with backtracking engines, how to write safer patterns, and which engines do not have the problem.
Why a Kafka client takes a minute to open a cluster with thousands of topics
Metadata, ListOffsets and message counts: what a client really asks for when you open an Apache Kafka cluster with thousands of topics and tens of thousands of partitions, and how to get under a second.