Spring Cloud Stream Using Kafka

Recently I've been digging in event-driven architectures for service communication. For demo followed broker approach instead of Feed approach, as broker approach has dedicated broker responsible for distributing the events to n number of subscribers.

For demo we will use most popular open-source Kafka broker (https://kafka.apache.org).

As spring (http://spring.io)  always makes life easy 😎, Spring-Cloud-Stream project by spring makes Kafka integration in your application very easy 👍 with few annotations.

Spring Cloud Stream provides Binder implementations for Kafka, Rabbit MQ, Redis, and Gemfire.

Note: Before we proceed make sure you have installed ZooKeeper and Kafka & both are running.

---

Dependencies Add following dependencies to your POM for using KAFKA.

---

Use Case: We will have One producer (demo-producer application) & two consumers (demo-consumer application & demo-consume1 application).

Producer raises event on Message Broker on Topic "demochannel",  Both Consumers has subscribed on Message Broker on Same Topic "demochannel" to consume event. This means as soon as event is raised, broker will notify respective subscriber to take action.

---

Producing Event: Create Spring Boot Project(demo-producer) with above dependencies. Create Main Application class : DemoProduceApplication.java as below

Create SourceChannel Interface

The @Output annotation identifies an output channel, through which published messages leave the application.

Create MessageProducer which will put message every second.

Add the @EnableBinding annotation to your application to get immediate connectivity to a message broker. 

Apart from raising event every second, we will create event using REST. So define rest-controller

application.properties looks like this.

---

Consuming Events: Create Spring-Boot Project(demo-consumer) with above dependencies Create Main Application class: DemoConsumerApplication.java

Create Sink Interface

The @Input annotation identifies an input channel, through which received messages enter the application. application.properties

Follow exactly same step as demo-consumer (defined above) for creating demo-consumer2 project, Once all required files are created change application.properties as below ( need to change only server port)

Steps to Exceute

Start zookeeper which is installed - sh zkServer.sh start. 

Start Kafka Server: sh bin/kafka-server-start.sh config/server.properties

Start: Producer Application (demo-producer) 

Start Both Application (demo-consumer & demo-consumer1)

Check - Console for both consumer

Hit REST endpoint using CURL: Here you see once request is served it raise events and which is consume by two subscribers.

Cheers – Happy learning 🙂

Ketan Gote