Aiven


Aiven Kafka

1. Get your Lenses license key 

Sign up with Lenses and download your license key.

2. Sign up for Aiven and create a Kafka 

Aiven for Apache Kafka is a fully managed streaming platform, deployable in the cloud of your choice.

Start by signing up for an Aiven account. Follow the in-product instructions to launch a Kafka cluster within the Aiven user interface.

3. Run Lenses with Docker 

Run the following in a terminal:

docker run -e LENSES_PORT=8080 -p 8080:8080 lensesio/lenses

Wait for this message in the logs:

2021-12-16 18:17:04,322 INFO  [i.l.b.r.HttpServer] [ioapp-compute-6] Lenses http server started and listening to requests.

Now your Lenses is up and running.

4. Login 

Head to http://localhost:8080.

Login as admin:

  • User: admin
  • Password: admin
Lenses login page

5. Wizard Setup 

From the aiven console on the Connection information Apache Kafka tab select as an Authentication Method SASL.

SASL Credentials

Grab the Service URI, place it on the wizard Bootstrap Servers and select as Security Protocol SASL SSL with SASL Mechanism SCRAM-SHA-256.

On the JAAS Configuration update the username and password from the respective fields User and Password on the Aiven console.

org.apache.kafka.common.security.plain.PlainLoginModule required
username="[Your_Connection_Details_Username]"
password="[Your_Connection_Details_Password]"
Kafka endpoint

Trustore.jks 

Open a new CLI tab and execute the Aiven-cli command - in case of not having the Aiven-cli installed, you can get it from the Aiven website:

avn service user-kafka-java-creds <YOUR SERVICE NAME> --username <YOUR AIVEN EMAIL> -d <DIRECTORY TO PRODUCE THE FILE> --password <A PASSWORD FOR YOUR TRUSTORE>

The command above will create 3 files at the directory that we specified:

  • ca.per
  • ca.pem
  • client.trustore.jks

On the same directory run the command:

docker run \
    -v ${PWD}/ca.pem:/tmp/file.pem:ro \
    -v ${PWD}/:/output \
    -it --rm lensesio/lenses:5.5.0 /usr/bin/keytool \
    -importcert \
    -noprompt \
    -trustcacerts \
    -keystore /output/truststore.jks \
    -alias “aivenjks” \
    -file /tmp/file.pem \
    -storepass changeit \
    -storetype JKS

This will create a trustore.jks on the same directory, with a password of changeit.

Proceed by adding the trustore file on the wizard, with its password - changeit.

Click on Test Kafka Brokers. And you should get the message: Validation completed successfully.

Trustore File

Kafka Metrics (Optional) 

Add Jolokia integration to enable JMX Metrics 

From the Aiven platform interface select:

Service Integrations ➡ Jolokia ➡ Create new

Kafka Metrics

Provide the:

  • Enpoint name
  • Username
  • Password

Enable Metrics Integration 

From the Overview tab, scroll down to “Service Integrations”. Click on Manage Integrations From the popup select Jolokia - Use Integration - and then enable:

Kafka Metrics
Jolokia Enable Aiven

Back to the wizard step 3, select:

  • Jolokia
  • API Call Type: POST
  • URL: HTTPS
  • Add Port Number: 6733
  • Add jolokia

As shown in the figure:

Jolokia Metrics Setup

Then toggle on the Use Authentication option, and from Aiven Service grab the username and the Password from:

Service Integrations ➡ Jolokia ➡ Endpoint Name

as you defined above.

Kafka Metrics

Click on Test Broker Metrics. And you should get the message: Validation completed successfully.

6. License 

Add your Lenses license. Copy and paste the JSON file you downloaded from your email. Lenses will check that the license is valid.

License Validation

7. Boot 

Click on Save & Boot Lenses. Lenses will finish configuring itself.

Wizard Validation

8. Eureka! 

Lenses Dashboard

Next Steps 

Additional Kafka features 

Enable Schema Registry 

From Lenses Select

Schema Registry ➡ Create Connection

Enable Schema Registry

On the aiven platform enable Schema Registry:

Aiven Schema Registry

On the Overview tab select Schema Registry from the connection information and grab the Service URI:

Schema Registry URI

Paste that in the Nodes, and click on Test Schema Registry Connection.

Schema Registry Nodes

You should get a validation message similar to:

Schema Registry Connect Success

Click on Apply Changes.

Enable Kafka Connect 

Kafka Connect endpoint

Similarly to the Scema Registry grab the Service URI from the Aiven Platform from the Apache Kafka Connect tab.

Aiven Kafka Connect URI

Back to Lenses select select connectors and create Connection:

Kafka Connect Endpoint Create Connection

Give it a name, and on the workers paste the uri:

Kafka Connect Endpoint Connection

Click on Test Kafka Connect Connection and similar to Schema Registry, hit on Apply Changes.

--
Last modified: June 18, 2024