Kafka 4.3: the deprecations to plan for before Kafka 5.0, starting with the classic rebalance protocol

Kafka 4.3 is not a disruptive release on its own — 25 KIPs and over 600 commits since 4.2.0 [1]. Its importance is what it schedules: several items are now formally marked for removal in Kafka 5.0 [1], and teams that read the deprecation list now get a release cycle of warning instead of a surprise.

Marked for removal in Kafka 5.0

• streams-scala is deprecated and marked for removal in 5.0 (KIP-1244) [1].
• The group.coordinator.rebalance.protocols broker configuration is deprecated and marked for removal (KIP-1237) [1].
• The existing MirrorMaker metrics are deprecated in favour of new metric names (KIP-1280); dashboards and alerts built on the old names need updating [1].

The classic rebalance protocol starts its exit

KIP-1274 is phase one of removing the classic rebalance protocol from KafkaConsumer: a consumer started with it now logs a message recommending the new consumer rebalance protocol, which the classic protocol is due to give way to in a future release [1]. Separately, KIP-1251 improves member-epoch validation for consumer groups to avoid unnecessary fencing [1].

Operational additions worth knowing

• Cordoned log directories (KIP-1066): new partitions cannot be placed on a cordoned directory, which helps when scaling or decommissioning brokers [1].
• Partition size percentage metrics (KIP-1257) show how much of its maximum retention each partition currently uses [1].
• Share groups gain additional broker and group configurations (KIP-1240) [1], following their production-ready status in 4.2.
• OAuth client assertion support for the client_credentials grant (KIP-1258) [1].

What we ask

We ask Kafka engineers how they would find every consumer in an estate still on the classic rebalance protocol, and what changes for a consumer group when it moves. The answer separates people who track the release notes from people who will find out at 5.0.