Setup Apache Kafka on Windows System

Hi Learners,

Here, I am going to show how to set apache kafka on windows machine. Before starting let understand the basic terminology about Apache kafka.

Kafka-

Kafka is a distributed messaging system providing fast, highly scalable and redundant messaging through a publish-subscribe pattern for building a real-time data pipelines and streaming applications.

  • A stream of messages belonging to the same type is defined as a Topic and it is a category or feed name to which records are published
  • A Producer can be anyone who can produce messages to a Topic in Kafka.
  • A set of servers where the publishes messages are stored, called Brokers or Kafka Cluster.
  • A Consumer can subscribe to one or more Topics and consume the published Messages by pulling data from the Brokers.
Environment versions

Windows 10
Apache kafka_2.11-0.10.1.0

Download and un-extract the kafka tar file from here

kafka_2.11-0.10.1.0.tgz

Explore the kafka directory and go to the config directory

Change the server.properties and zookeeper.properties

Change the log file location property in server.properties file
Ex - D:\kafka\kafka_2.11-0.10.1.0\config\server.properties

Change the data directory location property in zookeeper.properties file
Ex - D:\kafka\kafka_2.11-0.10.1.0\config\zookeeper.properties

Check the list of windows batch files
Ex - :\kafka\kafka_2.11-0.10.1.0\bin\windows

Start the zookeeper server

D:\kafka\kafka_2.11-0.10.1.0\bin\windows\zookeeper-server-start.bat ..\..\config\zookeeper.properties


Start the Kafka Broker

D:\kafka\kafka_2.11-0.10.1.0\bin\windows\kafka-server-start.bat ..\..\config\server.properties


Create the Topic

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic DemoTopic

Check the List the Topics

kafka-topics.bat --list --zookeeper localhost:2181

Interactively and real-timely testing

1- Start the Producer

kafka-console-producer.bat --broker-list localhost:9092 --topic DemoTopic

2- Start the Consumer

kafka-console-consumer.bat --zookeeper localhost:2181 --topic DemoTopic

Hope you are able to setup apache kafka on windows machine

Reference: -
https://kafka.apache.org/