Kafka Protocol
— Apache Kafka Wire ProtocolThe Kafka protocol is a binary RPC protocol over TCP designed for extreme scale (millions of events per second). Rather than destructive queues, Kafka treats topics as append-only partitioned commit logs, allowing consumer groups to independently read records at their own pace using persistent offsets.
Interactive Session: How Kafka Protocol Works
Step through the packet exchange sequence between nodes.
Interactive Execution Sandbox
Interact with a live or clearly-labelled simulated instance of Kafka Protocol.
Threat Model & Attack Simulation
How this protocol is attacked in practice, what defenders can see, and which control stops it.
Threat model & attack simulation
SimulatedDefensive education. Every step runs in this page; no traffic is generated or sent.
Open broker access
InterceptionA Kafka broker without authentication exposes topics, consumer offsets and produced records.
Reach the broker
The attacker connects to the plaintext listener.
Clients from unapproved networks.
Data disclosure and downstream fraud.
Defences that matter
- SASL and TLS listeners
- Topic ACLs and quotas
- Network isolation and client authentication
This is a conceptual model for defenders, not an exploit toolkit. It contains no payloads, tooling or instructions for attacking systems you do not own.
Protocol Specification & Architecture
Key characteristics, real-world deployment patterns, and trade-offs.
Core Technical Features
- Append-only partitioned commit log architecture
- Zero-copy data transfer from disk straight to network sockets via OS page cache
- Batching of produce requests and fetch responses
- Consumer Group offset tracking stored in `__consumer_offsets` topic
Real-World Use Cases
- Real-time telemetry and clickstream ingestion
- Distributed event sourcing and CQRS patterns
- Real-time stream processing with Kafka Streams and Flink
Advantages & Strengths
- Unmatched throughput (millions of messages/sec)
- Horizontal partition scaling and long-term event retention
Limitations & Vulnerabilities
- Steep operational learning curve and partition rebalancing complexities