Skip to main content

Apache Kafka, A distributed Streaming Platform

Image
By



This blog will guide 

 1 - How to install Zookeeper on Linux/Mac.

 2 - How to install Apache Kafka.

 3 - Commands to produce and consume message on Kafka.

Installation of ZooKeeper
  • Download latest zoopkeeper from http://zookeeper.apache.org/releases.html.
  • Create opt/sotware folder in user directory, It can be any directory. I have extracted zookeeper  in /Users/ketangote/opt/zookeeper.
  • cd /Users/ketangote/opt/zookeeper/conf
  • cp zoo_sample.cfg zoo.cfg
  • Open zoo.cfg using nano or vi and update data folder location:   dataDir=/Users/ketangote/opt/zookeeper/data
  • Start zookeeper commands: sh zkServer.sh start
  • Once zookeeper is started, use below command for verification.
           telnet localhost 2181
  • Stop ZoopKeeper server : shZKServer.sh stop


 Installation Of KAFKA
  • Download latest Kafka https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.0/kafka_2.11-0.10.1.0.tgz
  • Extract folder in /opt folder.It can be any directory.
  • Edit config/server.properties as below listeners=PLAINTEXT://:9099 broker.id=1
  • Start serversh bin/kafka-server-start.sh config/server.properties
  • Stop server sh bin/kafka-server-stop.sh config/server.properties


 Commands to Produce and Consume message on Kafka
  • Produce a message using below command sh bin/kafka-console-producer.sh --broker-list localhost:9099 --topic topic..... Type some messages
  • Consume a message using below command sh bin/kafka-console-consumer.sh --bootstrap-server localhost:9099 --topic ouput --from-beginningThis will display all message which are produce.
Labels:

Comments

Post a Comment

Popular posts from this blog

Centralized configuration using Spring Cloud Config

By
In this blog we will be focusing on centralized configuration using  Spring Cloud Config  project. For single standalone application we can keep all required configuration bundle with application itself.  However, when we have more than one application, say in a microservice architecture, a better alternative would be to manage the configurations centrally. With the Config Server we have a central place to manage external properties for applications with support for different environments. Configuration files in several formats like YAML or properties are added to a Git repository. Features Spring Cloud Config Server features: HTTP, resource-based API for external configuration (name-value pairs, or equivalent YAML content) Encrypt and decrypt property values (symmetric or asymmetric) Embeddable easily in a Spring Boot application using  @EnableConfigServer Config Client features (for Spring applications): Bind to the Config Server and initialize...
Post a Comment
Read more

Redis Basic CRUD

By Anonymous
We have seen how to setup on your linux machine here , now we will see how to perform basic CRUD operations using Spring Data & Redis server We will be creating a simple application that would persist an employee information into redis database. You will be needing certain JARs like jedis.jar, spring-data-redis.jar etc details of which you can download and view at https://github.com/meta-magic/RedisCRUDexample.git  First of all we will be creating the Employee entity, plz note that we are using the Serializable interface which automatically mapps the objects against the key. Employee.java import java.io.Serializable ; public class Employee implements Serializable { private static final long serialVersionUID = - 8243145429438016231L ; public enum Gender { Male , Female } private String id; private String name; private Gender gender; private int age; public String getId () { return id; } public void setId ( Str...
Post a Comment
Read more

Eureka-Server with spring cloud netflix

By
In this write-up we will focus on Service Registry – Eureka Server Rest service (auth-service application, eureka client) which register itself to registry. Web application which consumes Rest service using service registry. Service discovery  allows services to find and communicate with each other without hardcoding hostname and port. Eureka Server In spring-boot application enable the Eureka-Server by adding @EnableEurekaServer annotation to spring boot application. We have put all the configuration on GIT and this is accessed using config-server. To now more about centralized configuration (config-server) click  here eurekaserver.yml Place below bootstrap.yml in application, it basically connects to config-server and gets it required configuration. Start the spring-boot application and access eureka server using http://localhost:8760/ you will get below screen. Right now there is no application which...
Post a Comment
Read more