The previous article was about Brokers. In this article, we will see Topic and Partitions

In the above figure, broker b3 is receiving messages from both Sensor Application and Adserver Application. How does it differentiate between sensor messages and ad messages? Ignore consumer as of now. …

Previous article was about Topic and Partition. In the article, we will see Kafka Partition

Replication

  1. As shown in the above figure, the sensor topic is created with 3 partitions and 3 replications.
  2. 3 replications mean each partition will be replicated 3 times.
  3. Out of 3 replicas, 1 replica is designated…

In the previous article, we saw Kafka Replication. In this article, we will see Controller Broker

  1. Controller Broker kind of acts as the master of all the brokers. It also acts as a normal broker. i.e. Leader and Follower broker to transmit messages to and from Producer and Consumer respectively.

Brokers

  1. Brokers are the central workhorses of the Kafka cluster.
  2. Brokers are the JVM daemons running in the Kafka cluster.
  3. Always configure a single broker per node.
  4. Brokers receive messages from client applications (Producer) and send messages to processing application(consumer) on request.
  5. Brokers append all the messages received from producers to…

Components

  1. Kafka Message Queue
  2. Kafka Connect
  3. Kafka Streams
  4. Schema Registry
  5. Rest Proxy
  6. Embedded Zookeeper

Kafka Message Queue

Kafka Message Queue is a distributed circular persistent message queue. It is the core component of Kafka Stack. Kafka Connect and Kafka Streams internally use Kafka Message Queue.

Kafka Connect

Kafka Connect is used to transfer data from one…

This is a series of articles on Kafka. Lets start will a brief introduction to Kafka

Introduction to Kafka

  1. Kafka is a distributed circular persistent message queue.
  2. A queue is divided and distributed across the nodes in the cluster.
  3. Old messages are deleted after a certain time to make room for new messages…

Pramod Narayana

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store