Running Kafka for a streaming collection service can feel somewhat opaque at times, this is why I was thrilled to find the Kafka Web Console project on Github yesterday. This Scala application can be easily downloaded and installed with a couple steps. An included web server can then be launched to serve it up quickly. Here’s how to do all that.
For a quick intro to what the web console does, see my video first. Instructions for getting started follow below, including the quick video.
Kafka Web Console Project – Download
The main repository for this project is available on Github (claudemamo/kafka-web-console). However, I wanted the ability to add and remove Kafka topics so I use a forked repository that has a specific branch with those capabilities. (These are planned to be added to the main project but have not been yet.)
Download the ZIP archive file and unzip.
Before doing anything further, we need another application to build and launch the console.
Download Play Framework
A program called Play with Activator is used to build and launch the web console. It’s a Java app for launching Scala apps.
Download it here, unzip it and add it to the system path so you can execute the activator command that it provides.
Now back to the Kafka web console code. Enter the top level directory and execute the Play Activator start command (with one special option):
activator start -DapplyEvolutions.default=true
[info] Loading project definition from /home/demo/src/tmp/kafka-web-console-topic-add-remove/project
[info] Set current project to kafka-web-console (in build file:/home/demo/src/tmp/kafka-web-console-topic-add-remove/)
(Starting server. Type Ctrl+D to exit logs, the server will remain in background)
Play server process ID is 8528
[info] play - database [default] connected at jdbc:h2:file:play
[info] play - Starting application default Akka system.
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
The first time it runs will take some time to build the source and then launch the web server.
In the last line, above, you see it launches by default on port 9000.
Configuring the Console
Now you can launch the web console and start using the application. Step 1 figure shows the basic form for your Zookeeper configuration details. This is the only setup step required to get access to your Kafka brokers and topics.
The remained of the steps/figures are just showing the different screens. See the video for watching it in action.
- Diving into #NoSQL from the SQL Empire … - February 28, 2017
- VID: Solving Performance Problems on Hadoop - July 5, 2016
- Storing Zeppelin Notebooks in AWS S3 Buckets - June 7, 2016
- VirtualBox extension pack update on OS X - April 11, 2016
- Zeppelin Notebook Quick Start on OSX v0.5.6 - April 4, 2016
- Spark Analysis of Global Place Names (GeoNames) - January 20, 2016
- Serverspec checks settings on a Hadoop cluster - December 8, 2015
- Hadoop Options for SQL Databases - October 15, 2015
- “Big Data” off 2015 Hype Cycle? - August 18, 2015
- Spatial Data Made Useful - May 29, 2015