Skip to main content

Apache Kafka, A distributed Streaming Platform

Image
By



This blog will guide 

 1 - How to install Zookeeper on Linux/Mac.

 2 - How to install Apache Kafka.

 3 - Commands to produce and consume message on Kafka.

Installation of ZooKeeper
  • Download latest zoopkeeper from http://zookeeper.apache.org/releases.html.
  • Create opt/sotware folder in user directory, It can be any directory. I have extracted zookeeper  in /Users/ketangote/opt/zookeeper.
  • cd /Users/ketangote/opt/zookeeper/conf
  • cp zoo_sample.cfg zoo.cfg
  • Open zoo.cfg using nano or vi and update data folder location:   dataDir=/Users/ketangote/opt/zookeeper/data
  • Start zookeeper commands: sh zkServer.sh start
  • Once zookeeper is started, use below command for verification.
           telnet localhost 2181
  • Stop ZoopKeeper server : shZKServer.sh stop


 Installation Of KAFKA
  • Download latest Kafka https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.0/kafka_2.11-0.10.1.0.tgz
  • Extract folder in /opt folder.It can be any directory.
  • Edit config/server.properties as below listeners=PLAINTEXT://:9099 broker.id=1
  • Start serversh bin/kafka-server-start.sh config/server.properties
  • Stop server sh bin/kafka-server-stop.sh config/server.properties


 Commands to Produce and Consume message on Kafka
  • Produce a message using below command sh bin/kafka-console-producer.sh --broker-list localhost:9099 --topic topic..... Type some messages
  • Consume a message using below command sh bin/kafka-console-consumer.sh --bootstrap-server localhost:9099 --topic ouput --from-beginningThis will display all message which are produce.

Labels

Labels:

Comments

Post a Comment

Popular posts from this blog

Centralized configuration using Spring Cloud Config

By
In this blog we will be focusing on centralized configuration using  Spring Cloud Config  project. For single standalone application we can keep all required configuration bundle with application itself.  However, when we have more than one application, say in a microservice architecture, a better alternative would be to manage the configurations centrally. With the Config Server we have a central place to manage external properties for applications with support for different environments. Configuration files in several formats like YAML or properties are added to a Git repository. Features Spring Cloud Config Server features: HTTP, resource-based API for external configuration (name-value pairs, or equivalent YAML content) Encrypt and decrypt property values (symmetric or asymmetric) Embeddable easily in a Spring Boot application using  @EnableConfigServer Config Client features (for Spring applications): Bind to the Config Server and initialize...
Post a Comment
Read more

Function Point Analysis : ISO/IEC 20926:2009

By
This blog focuses on explaining the Function Point calculations. Software Sizing Background Function Point Rules for Counting FP Deep Dive - Function Point Analysis Case Study General Software Characteristics Details History - Measurement Methodologies Lines of Code (Oldest) Use case based Software Sizing IPFUG Function Point Analysis (ISO) Need for Software Sizing. Estimation and Budgeting Phasing Development Work Prioritization of Work Monitoring the Progress Bidding for Projects Allocating Testing Resources To measure and Manage Productivity Risk Assessment Software Asset Valuation CMMi Level 2 and 3 require that a valid sizing method be used. Software Sizing - Lines of Code The easiest and historically the most common method in Sizing Software project has been counting the number of lines of code and / or the number of screens. Advantages Automation of the counting process can be done Intuitive as the measurements are easily u...
Post a Comment
Read more

CRUD in MongoDB & Spring Data

By Anonymous
Now, since we have already been introduced to mongo and its server is setup. We shall now move on to performing basic CRUD operations. Lets take a use case example of ‘Company’ database that stores employee information.  We wish to store employee name, email address, age and multiple addresses. Traditionally in RDBMS we would create an Employee table and Address Table having foreign key reference to ‘employee id ‘ Incase of NoSQL, we will be creating Employee documnet which will have store employee information like name, email, age and an array of employes address. Following is a snippet of the schema defined { "name" : "", "email" : "", "age" : “”, "address" : [ ... ] } Thus to begin with, we will first define the entities. We have employee as an aggregate root entity that stores list of address having 1-M relatioship. Address Entity is represend as @Embeddable as it is embaded in another aggregate root entity. Employee is...
Post a Comment
Read more