Kafka: Quick Start on Mac

This post shows some examples for Kafka usages on Mac.


what is Kafka?

Apache Kafka is a distributed messaging system developed by LinkedIn.


install

% java --version
openjdk 11.0.10 2021-01-19
OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.10+9)
OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.10+9, mixed mode)
% brew install kafka
% kafka-topics --version                                                                             
2.8.0 (Commit:ebb1d6e21cc92130)

Note: zookeeper is also installed


start

You have two options to start Kafka, as shown in the installation message:

To have launchd start kafka now and restart at login:
  brew services start kafka
Or, if you don't want/need a background service you can just run:
  zookeeper-server-start -daemon /usr/local/etc/kafka/zookeeper.properties & kafka-server-start /usr/local/etc/kafka/server.properties


create topic

% kafka-topics --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test2
Created topic test2.
% kafka-topics --list --zookeeper localhost:2181
test2


produce & consume

consumer

% kafka-console-consumer --bootstrap-server localhost:9092 --topic test2

producer

% kafka-console-producer --broker-list localhost:9092 --topic test2

You can see what you input on the producer side is reflected on the consumer side.




delete topic

You need to enable set delete.topic.enable to true, before deleting a topic.

% vim /usr/local/etc/kafka/server.properties
delete.topic.enable=true
% kafka-topics --delete --zookeeper localhost:2181 --topic test2
Topic test2 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.


Comments

Post a Comment

Popular posts from this blog

Selenide: Quick Start

Selenide is a Selenium wrapper in Java for testing.   This post describes how you can start using Selenide (using Gradle). 1. Setup Java Application # keep pressing enters to choose default values gradle init --type java-application # confirm that test succeeds gradle test 2. Add Selenide Dependency open build.gradle and edit as: dependencies {   ...   testImplementation 'com.codeborne:selenide:5.13.0' } 3. Write Test (replace the existing test with selenide code) src/test/java/selenide/AppTest.java package selenide; import static com.codeborne.selenide.Condition.text; import static com.codeborne.selenide.Selenide.$; import static com.codeborne.selenide.Selenide.open; import org.junit.Test;   public class AppTest { @Test public void test() { open("https://www.google.ca/"); $("input[type=text]").val("test").pressEnter(); $("body").shouldHave(text("test")); } } 4. Run Selenide gradle test This will open a browser t...
Read more

Minikube Installation for M1 Mac

Image
In this post, I am going to try Minikube on Mac with M1 Chip. what is minikube Minikube is k8s tool running on a local machine. See also kind  for multi cluster implementation for local k8s. install % curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-arm64 % install minikube-darwin-arm64 /usr/local/bin/minikube % minikube version  minikube version: v1.20.0  Note: Installing with brew fails due to a different binary as of 2021-05-24. % brew install minikube % minikube version ❌  Exiting due to MK_WRONG_BINARY_M1: You are trying to run amd64 binary on M1 system. Please use darwin/arm64 binary instead (Download at https://github.com/kubernetes/minikube/releases/download/v1.20.0/minikube-darwin-amd64.) % brew uninstall minikube start The default driver 'virtualbox' is not supported on M1, but you can use 'docker' driver:  https://minikube.sigs.k8s.io/docs/drivers/docker/  . Make sure docker is running, and then specify the driver ...
Read more

Server Testing Tools: Serverspec, InSpec, Testinfra, Goss

In this post, I am going to describe differences among server-testing frameworks that I have briefly tried so far: Comparison Serverspec InSpec Testinfra Goss my blog post link link link link language Ruby / RSpec Ruby / RSpec Python / pytest Go / yaml package gem gem pip none remote exec yes yes yes no license MIT Chef Apache 2.0 Apache 2.0 github repo link link link link - created_at 2013 2015 2015 2015 - stars 2.4k 2.3k 1.9k 4.3k Thought Personally, I liked Goss the most as you can write it in yaml. The biggest drawback is that you have to run it on the server itself, and this makes it harder to introduce the tool on the existing production environment. Next, InSpec was easy to start with. I was going to use it at work until I noticed the license issue. Thus, I have basically two options. Serverspec seems great, but since my team members tends to prefer Python to other languages, we might start using Testinfra first.
Read more