Lenses Box

Lenses Box is a container solution for building applications on a localhost Apache Kafka docker.

Start your DataOps Experiencelensesio

Lenses Box download

What’s in the Box?

Lenses Box contains all components of the Apache Kafka ecosystem, CLI tools, and synthetic data streams.

Lenses Box

Starting the Box

1. To start with the Box get your free development license online.

2. Install and run the Docker

docker run --rm \
    -p 3030:3030 \
    --name=dev \
    --net=host \
    -e EULA="https://dl.lenses.stream/d/?id=CHECK_YOUR_EMAIL_FOR_KEY" \
    lensesio/box   

Open Lenses in your browser, log in with admin/admin.

Kafka Docker advertisement

The broker in the Kafka docker has a broker id 101 and advertises the listener configuration endpoint to accept client connections.

If you run Docker on macOS or Windows, you may need to find the address of the VM running Docker and export it as the advertised listener address for the broker (On macOS it usually is 192.168.99.100). At the same time, you should give the lensesio/box image access to the VM’s network:

docker run -e EULA="CHECK_YOUR_EMAIL_FOR_KEY" \
           -e ADV_HOST="192.168.99.100" \
           --net=host --name=dev \
           lensesio/box

If you run on Linux, you don’t have to set the ADV_HOST , but you can do something cool with it. If you set it to be your machine’s IP address, you can access Kafka from any clients in your network.

If you decide to run a box in the cloud, you (and all your team) can access Kafka from your development machines. Remember to provide the public IP of your server as the kafka advertised host for your producers and consumers to access it.

Kafka Docker JMX

Kafka JMX metrics are enabled by default. Refer to ports once you expose the relevant port, ie. -p 9581:9581 you can connect to JMX with

jconsole localhost:9581

Custom hostname

If you are using docker-machine or setting this up in a Cloud or DOCKER_HOST is a custom IP address such as 192.168.99.100, you will need to use the parameters --net=host -e ADV_HOST=192.168.99.100.

docker run --rm \
    -p 3030:3030 \
    --net=host \
    -e ADV_HOST=192.168.99.100 \
    -e EULA="https://dl.lenses.stream/d/?id=CHECK_YOUR_EMAIL_FOR_KEY" \
    lensesio/box    

Docker data persistence

To persist the Kafka data between multiple executions, provide a name for your Docker instance and do not set the container to be removed automatically (--rm flag). For example:

docker run \
    -p 3030:3030 -e EULA="CHECK_YOUR_EMAIL_FOR_KEY" \
    --name=dev lensesio/box

Once you want to free up resources, just press Control-C. Now you have two options: either remove the Docker:

docker rm dev

Or use it at a later time and continue from where you left off:

docker start -a devhshel

Port Numbers

Service	Port Number
Kafka broker	9092
Kafka connect	8083
Zookeeper	2181
Schema Registry	8081
Lenses	3030
Elasticsearch	9200
Kafka broker JMX	9581
Schema registry JMX	9582
Kafka connect JMX	9584
Zookeeper JMX	9585
Kafka broker (ssl)	9092

Advanced options

Variable	Description
ADV_HOST=[ip-address]	The ip address that the broker will advertise
DEBUG=1	Prints all stdout and stderr processes to container’s stdout for debugging.
DISABLE_JMX=1	Disables exposing JMX metrics on Kafka services.
ELASTICSEARCH_PORT=0	Will not start Elasticsearch.
ENABLE_SSL=1	Creates CA and key-cert pairs and makes the broker also listen to SSL://127.0.0.1:9093
KAFKA_BROKER_ID=1	Overrides the broker id (the default id is 101).
SAMPLEDATA=0	Disables the synthetic streaming data generator that are running by default.
SUPERVISORWEB=1	Enables supervisor interface on port 9001 (adjust via SUPERVISORWEB_PORT) to control services.

FAQ

How can I run offline?

Download your key locally and run the command:

LFILE=`cat license.json`
docker run --rm -it -p 3030:3030 -e LICENSE="$LFILE" lensesio/box:latest

How much memory to allocate?

The container is running multiple services, and it is recommended to allocate 5GB of RAM to the docker (although it can operate with even less than 4GB).

To reduce the memory footprint, it is possible to disable some connectors and shrink the Kafka Connect heap size by applying these options (choose connectors to keep) to the docker run command:

-e DISABLE=azure-documentdb,cassandra,elastic5,ftp,influxdb,jms,mongodb,mqtt,redis
-e CONNECT_HEAP=512m