Kafka 4.0 removed ZooKeeper. What the upgrade actually involves

Kafka 4.0 is the first major release to run entirely without ZooKeeper [1]. For teams still on a ZooKeeper-based cluster, that turns what used to be an optional modernisation into a prerequisite. Here is what the upgrade involves, in roughly the order it tends to bite.

ZooKeeper clusters migrate first

Kafka 4.0 supports only KRaft mode; ZooKeeper mode has been removed [2]. A cluster still running ZooKeeper has to be migrated to KRaft before it can be upgraded to 4.0 [2]. Broker upgrades to 4.0 also require software and metadata versions of at least 3.3, the first release in which KRaft was considered production ready, and the documentation recommends that KRaft clusters older than that move to 3.9 first [2].

In practice the 4.0 upgrade is two projects, not one. The KRaft migration is the hard part — it changes how the cluster's metadata is managed — and it deserves to be planned, rehearsed and stabilised on its own before the version jump.

Java and protocol floors

• Java 17 for brokers, Connect and the tools; Java 11 for clients and Kafka Streams [1]. Runtime upgrades on broker hosts are often the slowest item, because they touch build images and change approval rather than Kafka itself.
• Brokers and clients must each be on 2.1 or later before the other side moves to 4.0 [1]. Old clients embedded in rarely touched services are the usual surprise.
• Message formats v0 and v1 are removed [1], having been deprecated since 3.0. Anything still producing or depending on them has to be found before the upgrade, not after.

The new rebalance protocol is opt-in

4.0 makes the next-generation consumer rebalance protocol generally available — the change the project describes as saying goodbye to stop-the-world rebalances [1]. It is enabled by default on the server, but consumers have to opt in by setting `group.protocol=consumer` [1]. That makes it a separate, deliberate rollout: worth doing, but not something that happens to you on upgrade day.

The wider point

Major version upgrades on messaging platforms rarely fail on the headline change. They fail on the prerequisites nobody inventoried: the service still on an ancient client, the host image still on an old JDK, the consumer nobody owns. The useful skill is not knowing what changed in 4.0. It is knowing to go and find those before the window opens.