Kafka share groups are production-ready in 4.2: does Kafka replace your queue now?

For most of its life, Kafka's answer to work-queue semantics was 'add partitions': a consumer group can never have more active consumers than a topic has partitions. KIP-932 changes that by adding a new kind of group.

Three releases, three stages

What a share group is

• A share group enables cooperative consumption of Kafka topics [1]; the project describes it as roughly equivalent to a durable shared subscription in other systems [1].
• Multiple consumers can concurrently process records from the same partitions, with individual acknowledgement and delivery counting, without strict partition-to-consumer assignment [3].
• The 4.2 release added a RENEW acknowledgement type for long processing times, and share partition lag metrics for monitoring consumption [3].

Does it replace RabbitMQ or SQS?

For teams already running Kafka who kept a second broker only for work queues, share groups remove the main reason for it. For teams whose queues lean on routing, priorities or per-message TTL, the broker is doing more than cooperative consumption, and a share group covers only part of it. The question is not which is better; it is which features the existing queue is actually being used for.

What we ask

We ask Kafka engineers when they would choose a share group over a consumer group, and what they would check about ordering before switching. The answer tells us whether they think in partitions or in delivery guarantees.