Instaclustr

Instaclustr

Get started with Lenses. Set it up with Docker and connect to an Instaclustr managed Kafka with the standard configuration.

1. Get your Lenses license key 

Sign up with Lenses and download your license key.

2. Sign up for Instaclustr and create a Kafka 

Instaclustr Apache Kafka is a fully managed streaming platform, deployable in the cloud of your choice.

Start by signing up for an Instaclustr account. Follow the in-product instructions to launch Kafka clusters within the Instaclustr user interface.

Then, enable connectivity to your Kafka.

Enable firewall access 

Add your Lenses IP address to the firewall lists:

  1. Kafka Cluster
  2. Schema Registry
  3. Kafka Connect

Go to Settings on your Kafka and Kafka Connect clusters.

Kafka firewall settings
Kafka Connect firewall settings

Gather 3 connection points 

  1. Kafka endpoint. Go to Connection Info

Kafka endpoint
Kafka authentication details
2. Schema Registry endpoint.

Schema Registry endpoint
3. Kafka Connect endpoint & certificates. Go to your Connect Cluster Connection Info

Kafka Connect endpoint
Kafka Connect authentication details

Download the Connect certificates. Unzip them.

Kafka Connect certificates

3. Get Docker 

You will use docker-compose to spin up Lenses.

Start by signing up with Docker. Then install docker-compose.

Install Docker

4. Setup Lenses docker-compose 

Use this docker-compose.yml file to spin up Lenses and connect to your Instaclustr Kafka.

Just 2 steps:

  1. Fill in the connection details:
    1. Kafka endpoint
    2. Schema Registry
    3. Kafka Connect
  2. Share the authentication files with your docker. Copy these files in the same directory as your docker-compose.yml file and reference them:
    1. Your Lenses license key license.json
    2. The Instaclustr Connect certificate store truststore.jks and its password - find it in the README file.

The file:

version: '3'
services:
  lenses:
    image: lensesio/lenses
    environment:
      # Admin user credentials
      LENSES_SECURITY_USER: admin
      LENSES_SECURITY_PASSWORD: admin
      LENSES_PORT: 8080
      
      # === Set your values in this section ===
            
      # ===Step 1=== Put your Instaclustr Kafka 'Node Addresses' here, port 9092
      # e.g. 
      # LENSES_KAFKA_BROKERS: 'SASL_PLAINTEXT://18.134.119.76:9092,SASL_PLAINTEXT://18.134.23.61:9092,SASL_PLAINTEXT://18.134.47.193:9092'
      # Kafka Endpoint
      LENSES_KAFKA_BROKERS: 'SASL_PLAINTEXT://[KAFKA_HOST_1]:[PORT_1],SASL_PLAINTEXT://[KAFKA_HOST_2]:[PORT_2],…'
      # Instaclustr SSL configuration
      LENSES_KAFKA_SETTINGS_CLIENT_SECURITY_PROTOCOL: SASL_PLAINTEXT
      LENSES_KAFKA_SETTINGS_CLIENT_SASL_MECHANISM: SCRAM-SHA-256
  
      # ===Step 2=== Put the Kafka username and password here for the Kafka Client to access Instaclustr Kafka
      FILECONTENT_JAAS: |
        KafkaClient {
          org.apache.kafka.common.security.scram.ScramLoginModule required
          username="[INSTA_USERNAME]"
          password="[INSTA_PASSWORD]"
          serviceName=kafka;
        };        


      # ===Step 3=== Put the Schema Registry endpoint here
      # Instaclustr Schema Registry
      LENSES_SCHEMA_REGISTRY_URLS: '[{url: "https://[REGISTRY_HOST]:[REGISTRY_PORT]"}]'
      LENSES_SCHEMA_REGISTRY_AUTH: USER_INFO
      LENSES_SCHEMA_REGISTRY_USERNAME: [USER]
      LENSES_SCHEMA_REGISTRY_PASSWORD: [PASSWORD]
      LENSES_KAFKA_SETTINGS_CLIENT_BASIC_AUTH_CREDENTIALS_SOURCE: USER_INFO
      LENSES_KAFKA_SETTINGS_CLIENT_BASIC_AUTH_USER_INFO: [USER]:[PASSWORD]
      
      # ===Step 4=== Put the Kafka Connect endpoint here
      # Instaclustr Kafka Connect
      LENSES_KAFKA_CONNECT_CLUSTERS: |
        [{
          name:"instaclustr-connect",
          urls: [{url:"https://[CONNECT_HOST_1]:[PORT_1]"},{url:"[CONNECT_HOST_2]:[PORT_2]"},…],
          auth: "USER_INFO",
          username: "[USER]",
          password: "[PASSWORD]",
          statuses: "__connect_status",
          configs: "__connect_configs",
          offsets: "__connect_offsets"
        }]        

      # ===Step 5=== Put your Kafka Connect certificate password here, 2x times
      LENSES_KAFKA_CONNECT_SSL_KEYSTORE_LOCATION: /mnt/secrets/connectstore.jks
      LENSES_KAFKA_CONNECT_SSL_KEYSTORE_PASSWORD: [CONNECT_CERT_PASSWORD]  
      LENSES_KAFKA_CONNECT_SSL_TRUSTSTORE_LOCATION: /mnt/secrets/connectstore.jks
      LENSES_KAFKA_CONNECT_SSL_TRUSTSTORE_PASSWORD: [CONNECT_CERT_PASSWORD]

    ports:
      - 8080:8080
    volumes:
      # ===Step 6=== Share your authentication files here
      # Map the Lenses license key file here to activate Lenses
      - ./license.json:/license.json
      # Map the Instaclustr truststore here to enable Lenses to communicate with Kafka Connect via SSL
      - ./truststore.jks:/mnt/secrets/connectstore.jks

5. Start Lenses up 

docker-compose up

6. Login 

Open your browser and go to:

http://localhost:8080

Login as admin:

User:     admin
Password: admin

7. Eureka! 

Lenses Dashboard

Next steps 

Limitations 

With Instaclustr, this endpoint is not available:

  • No Kafka metrics via JMX

Without this endpoint these Lenses features are not available:

--
Last modified: April 17, 2024