Okay, so I’ve been messing around with Kafka lately, and specifically trying to figure out which consumer group monitoring tool to use. It came down to two main contenders: Burrow and Kafka-Manager (formerly Yahoo Kafka Manager, now I guess just called “CMAK”). I kept seeing these two names pop up, so I decided to get my hands dirty and see which one worked better for my needs.
Setting Things Up
First, I spun up a simple Kafka cluster. Nothing fancy, just enough to get some data flowing. I already had Zookeeper and Kafka running locally, so that part was easy. Then, I created a couple of topics and started pumping some dummy messages into them using a simple producer script I whipped up.
Trying Out Burrow
Next up was Burrow. I grabbed the latest release from their GitHub, extracted the files, and started poking around. The configuration seemed a little daunting at first, but the documentation was pretty decent. I had to set up a config file () and point it to my Kafka and Zookeeper instances. I spent some time tweaking the settings, figuring out which storage option to use (I went with the in-memory one for simplicity), and setting up the HTTP server section so I could access the UI.
Once I got Burrow running, I pointed my browser to the status endpoint. Boom! There was a nice overview of my consumer groups, showing their lag and status. It was pretty cool to see everything laid out like that. The UI felt a bit basic, but it did the job. I liked that it gave me a clear picture of how far behind my consumers were.
Taking a Look at Kafka-Manager (CMAK)
Then I moved on to Kafka-Manager. I downloaded the latest release, extracted it, and similar to Burrow, I had to configure it. The main config file was , and I had to set the Zookeeper hosts there. After a bit of fiddling, I got it up and running. The interface was definitely more polished than Burrow’s. I could see all my topics, brokers, and consumer groups in a much more organized way. It also had some extra features, like the ability to reassign partitions and manage topics, which Burrow didn’t have. It felt more like a full-fledged management tool.
The Verdict?
So, after playing around with both, here’s my take:
- Burrow: Simpler to set up for basic consumer group monitoring. Good for quickly checking lag and status. The UI is functional but not fancy.
- Kafka-Manager (CMAK): More comprehensive management features. Better UI, but a bit more complex to configure. Offers more than just consumer group monitoring.
For my immediate needs, which were primarily focused on monitoring consumer lag, Burrow actually felt like a better fit. It was quicker to get going and gave me the information I needed without any extra fluff. However, I can see how Kafka-Manager would be super useful in a larger, more complex environment where you need those extra management capabilities. I think I will keep both of them. Since I might need CMAK in the future.
It really boils down to what you’re looking for. If you just want a simple, no-frills way to keep an eye on your consumer groups, Burrow is a solid choice. If you need a more powerful, feature-rich tool, Kafka-Manager is the way to go. I hope you can benefit from this.
