What is Kafka?
Kafka is a distributed, partitioned, replicated, log service developed by LinkedIn and open sourced in 2011. Basically it is a massively scalable pub/sub message queue architected as a distributed transaction log. It was created to provide “a unified platform for handling all the real-time data feeds a large company might have”.1
- As mentioned above, it is fundamentally a replicated log service.
- It does not use AMQP or any other pre-existing protocol for communication. Instead, it uses a custom binary TCP-based protocol.
- It is very fast, even in a small cluster.
- It has strong ordering semantics and durability guarantees.