Remoting over Apache Kafka with Kubernetes features

This site is the new docs site currently being tested. For the actual docs in use please go to https://www.jenkins.io/doc.

Goal: Enhance the plugin and to provide out-of-the-box management in Kubernetes environment

Status: Completed

Team

Student: longnguyen Long Nguyen

Mentor(s): afalko Andrey Falko pvtuan10 Pham Vu Tuan no image Ankit Jain no image Supun Wanniarachchi oleg nenashev Oleg Nenashev

Motivation

Previous version of Remoting over Apache Kafka plugin requires users to manually configure the entire system which includes Zookeeper, Apache Kafka and Remoting agents. It also didn’t support dynamic agent provisioning so scalability was harder to achieve. This project aims to solve two problems:

  1. Out-of-the-box solution to provision Apache Kafka cluster.

  2. Dynamic agent provisioning in a Kubernetes cluster.

Phase 1 Summary

Phase 2 Summary

Phase 3 Summary

Features

Here are some screenshots of the implemented features

1. Launch Apache Kafka on Kubernetes

Apache Kafka provisioning in Kubernetes UI

2. Cloud configuration

Remoting Kafka Cloud UI

3. Remoting Kafka Agent get provisioned dynamically

Remoting Kafka Agent get provisioned

4. Helm Chart to bootstrap the whole system on Kubernetes

Kubectl get all to check Helm chart installation