Skip to main content

Apache Kafka, A distributed Streaming Platform

Image
By



This blog will guide 

 1 - How to install Zookeeper on Linux/Mac.

 2 - How to install Apache Kafka.

 3 - Commands to produce and consume message on Kafka.

Installation of ZooKeeper
  • Download latest zoopkeeper from http://zookeeper.apache.org/releases.html.
  • Create opt/sotware folder in user directory, It can be any directory. I have extracted zookeeper  in /Users/ketangote/opt/zookeeper.
  • cd /Users/ketangote/opt/zookeeper/conf
  • cp zoo_sample.cfg zoo.cfg
  • Open zoo.cfg using nano or vi and update data folder location:   dataDir=/Users/ketangote/opt/zookeeper/data
  • Start zookeeper commands: sh zkServer.sh start
  • Once zookeeper is started, use below command for verification.
           telnet localhost 2181
  • Stop ZoopKeeper server : shZKServer.sh stop


 Installation Of KAFKA
  • Download latest Kafka https://www.apache.org/dyn/closer.cgi?path=/kafka/0.10.1.0/kafka_2.11-0.10.1.0.tgz
  • Extract folder in /opt folder.It can be any directory.
  • Edit config/server.properties as below listeners=PLAINTEXT://:9099 broker.id=1
  • Start serversh bin/kafka-server-start.sh config/server.properties
  • Stop server sh bin/kafka-server-stop.sh config/server.properties


 Commands to Produce and Consume message on Kafka
  • Produce a message using below command sh bin/kafka-console-producer.sh --broker-list localhost:9099 --topic topic..... Type some messages
  • Consume a message using below command sh bin/kafka-console-consumer.sh --bootstrap-server localhost:9099 --topic ouput --from-beginningThis will display all message which are produce.
Labels:

Comments

Post a Comment

Popular posts from this blog

Redis Basic CRUD

By Anonymous
We have seen how to setup on your linux machine here , now we will see how to perform basic CRUD operations using Spring Data & Redis server We will be creating a simple application that would persist an employee information into redis database. You will be needing certain JARs like jedis.jar, spring-data-redis.jar etc details of which you can download and view at https://github.com/meta-magic/RedisCRUDexample.git  First of all we will be creating the Employee entity, plz note that we are using the Serializable interface which automatically mapps the objects against the key. Employee.java import java.io.Serializable ; public class Employee implements Serializable { private static final long serialVersionUID = - 8243145429438016231L ; public enum Gender { Male , Female } private String id; private String name; private Gender gender; private int age; public String getId () { return id; } public void setId ( String
Post a Comment
Read more

Function Point Analysis : ISO/IEC 20926:2009

By
This blog focuses on explaining the Function Point calculations. Software Sizing Background Function Point Rules for Counting FP Deep Dive - Function Point Analysis Case Study General Software Characteristics Details History - Measurement Methodologies Lines of Code (Oldest) Use case based Software Sizing IPFUG Function Point Analysis (ISO) Need for Software Sizing. Estimation and Budgeting Phasing Development Work Prioritization of Work Monitoring the Progress Bidding for Projects Allocating Testing Resources To measure and Manage Productivity Risk Assessment Software Asset Valuation CMMi Level 2 and 3 require that a valid sizing method be used. Software Sizing - Lines of Code The easiest and historically the most common method in Sizing Software project has been counting the number of lines of code and / or the number of screens. Advantages Automation of the counting process can be done Intuitive as the measurements are easily u
Post a Comment
Read more

Redis Installation Steps

By Anonymous
“ Redis is an in-memory key-value store known for its flexibility, performance, and wide language support” Inorder to install redis on your machine you need ubuntu 16.4 and a non-root user with sudo privileges to perform the administrative functions required for this process. Download and Extract the Source Code Create a tmp directory cd /tmp Download the latest stable version of Redis curl -O http://download.redis.io/redis-stable.tar.gz untar tar xzvf redis-stable.tar.gz Move into the Redis source directory structure that was just extracted cd redis-stable Build and Install Redis Now, we can compile the Redis binaries by typing make After the binaries are compiled, run the test suite to make sure everything was built correctly. You can do this by typing: make test This will typically take a few minutes to run. Once it is complete, you can install the binaries onto the system by typing: sudo
Post a Comment
Read more