Content deleted Content added
Ligaturama (talk | contribs) m →Connect API: Update tag |
m Add KIP-932: Queues for Kafka |
||
Line 31:
In contrast, Apache Kafka is a distributed log-based messaging system that guarantees ordering within individual partitions rather than across the entire topic. Unlike queue-based systems, Kafka retains messages in a durable, append-only log, allowing multiple consumers to read at different offsets. Kafka uses manual offset management, giving consumers control over retries and failure handling. If a consumer fails to process a message, it can delay committing the offset, preventing further progress in that partition while other partitions remain unaffected. This partition-based design enables fault isolation and parallel processing while allowing ordering to be maintained within partitions, depending on consumer handling.<ref>{{Cite book |last=Narkhede |first=Neha |title=Kafka: the definitive guide: real-time data and stream processing at scale |last2=Shapira |first2=Gwen |last3=Palino |first3=Todd |date=2017 |publisher=O'Reilly Media |isbn=978-1-4919-3616-0 |___location=Sebastopol, CA |oclc=933521388}}</ref>
In 2025, Apache Kafka introduced "Queues for Kafka" through KIP-932,<ref>[https://cwiki.apache.org/confluence/display/KAFKA/KIP-932%3A+Queues+for+Kafka KIP-932: Queues for Kafka]</ref> adding share groups as an alternative to consumer groups. This feature enables queue-like semantics where consumers can cooperatively process records from the same partitions, with individual message acknowledgment and delivery tracking. Unlike traditional consumer groups where partitions are exclusively assigned, share groups allow the number of consumers to exceed partition count, making it ideal for work-queue patterns while maintaining Kafka's durability and scalability benefits. This development addresses the common challenge of "over-partitioning" that many Kafka users face.
==Kafka APIs==
| |||