Kafka Learning

First step to install the tar file from the quickstart page and extract and open the cmd from the path which has the bin folder.

bin\windows\zookeeper-server-start.bat config\zookeeper.properties

Open another cmd from the same path and then run the server.

bin\windows\kafka-server-start.bat config\server.properties

Create a topic in kafka.

bin\windows\kafka-topics.bat --create --topic user-topic --bootstrap-server localhost:9092

We created the topic by the name user-topic.

List of the topic created.

We need –bootstrap-server r –zookeeper to see the list of the topic.

bin\windows\kafka-topics.bat --bootstrap-server=localhost:9092 --list

Other command.

bin\windows\kafka-topics.bat --list --bootstrap-server localhost:9092

Both giving the output user-topic.

Creating message inside the topic.

Consumer subscribed to the topic will get the message. We create the message in the producer.

bin\windows\kafka-console-producer.bat --topic user-topic --bootstrap-server localhost:9092

Here we can write the message.

Consuming the message.

bin\windows\kafka-console-consumer.bat --topic user-topic --from-beginning --bootstrap-server localhost:9092

Take the topic name properly. One can pen multiple instance of CMD and can subscribe the same topic and will get the message.

Making a Spring Project to use the Kafka Messaging Queue.

Making one Spring project to update the location and other to receive the update location message. One project by name endUser and other by the name location.

• For producing the message we need a topic. Create a config package for the Configuration class.

• Message updated when we hit the url with post request.

  

• From the cmd we can get the message.

• In the producer yml file, in location project we need to write.

#Producer configuration
spring.kafka.producer.bootstrap-servers=localhost:9092
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer

• In the consumer yml file, in the endUser project we need to write.

server.port=8081
spring.kafka.consumer.bootstrap-servers=localhost:9092
spring.kafka.consumer.group-id=group-id
spring.kafka.consumer.auto-offset-reset=earliest
spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

Bootstrap server running in confluent. To see if the bootstrap server is connected and active in the url.
In curl or command prompt curl -v telnet://lkc-1jr5dz-61m326.us-central1.gcp.glb.confluent.cloud:9092 It is connected. In confluent we can see the bootstrap server url It need vpn to connect.

In case it shows telnet not recognized. Enable Telnet client - Open Run - optionalfeatures.exe - Telnet client - Ok.

Learning About TelNet.

Telnet = Telecommunication Network. It is a protocol that allow a computer to connect to another computer over the TCP/IP and send text based command.
Traditionally it was used to connect remotely to server now SSH used. The command is used just to test if a TCP port is reachable.
telnet example.com 80 - It will try to connect to example.com on port 80. can my machine reach this host:port?
It cannot be used in kafka command. Kafka needs SASL_SSL authentication and binary protocol, which Telnet cannot handle.