This page describes how to authentication against Lenses APIs.
All requests must be authenticated using an HTTP Header x-kafka-lenses-token:myToken. You can obtain the token via the following login API or you can use a service account.
All REST APIs are protected via role-based authentication that is either BASIC or LDAP based, depending on how Lenses security has been set up. In order to be able to use the APIs, you will need to first authenticate via an appropriate user, then receive an access token and use that token for any subsequent request.
POST/api/login
Headers
| Name | Value |
|---|---|
Body
| Name | Type | Description |
|---|---|---|
Response
To run the example below, we recommend installing the jq tool
A WebSocket allows the client to request data using SQL as input.
Use the following URL to open a WebSocket connection. Depending on deployment, use ws (not-secure) or wss (secure) connection:
Once the connection is established, the server expects the client to send a JSON message with the following structure:
| Field | Description | Type |
|---|
The client should wait for incoming messages and stop when EOF is received. The server sends JSON-encoded messages, the payload structure is:
Since there are multiple types of messages the server sends, the type attribute determines the data structure for the data payload. Below is the list of possible type values:
The data the attribute is null. The client should ignore these messages.
In this case, the payload structure is:
When requested the payload format is:
A possible payload can be:
The payload format is:
The payload format is:
The payload format is:
The payload format is:
Given the type RECORD the payloads contains the data returned to the client, and it is dependent on the SQL input. The payload format is:
For a topic which contains nested data for both key and value.
The payload format is:
If the Kafka message key is null the response would be:
Projecting the key will not return the key attribute:
produces the following output:
Aggregating data will not return the key and metadata attributes:
| Value | Description |
|---|
| Value | Description |
|---|
Content-Type
application/json
user
string
Name of the user
password
string
Password of the user
Authenticate
Learn how to authenticate against Lenses APIs.
Lenses API Spec
Explore the Lenses API Spec.
Websocket Spec
Lenses uses websockets for SQL queries. Learn how to use Lenses Websockets.
Provisioning API
Explore the Provisioning API Spec.
HEARTBEAT | To keep the connection alive, the server injects an empty message. |
RECORD | The message represents a data record. |
ERROR | The message contains information about an error that occurred. |
STATS | The message contains information about the current execution.Only activated if the |
METADATA | The message contains a list of metadata fields each record message will contain. For example: offset, partition,timestamp, __keysize, __valueSize, valueSchemaId. |
SCHEMA | Reserved. |
PAGE_END | Reserved. |
END | The message signals the end of the execution. The server will close the socket after this message is sent. |
BADRECORD | The message contains information about the Kafka message which cannot be read. For example, if the topic expects Avro and the payload is not a valid Avro. |
SENTINEL | The message contains information about the execution termination reason when query thresholds are reached. |
data.key | Returns the underlying Kafka message key value. Only applied when |
data.data | Will contain the output generated by the SQL projections. If |
data.metadata | Returned when no aggregations are involved. It contains the Kafka message partition, offset, byte size information |
data.rownum | Reserved optional long value. |
token | How often to refresh kafka topic list and configs | long |
sql | The SQL query used to fetch data | string |
live | If set, enables partial results for aggregation queries. | optional[boolean] |
stats | Millis interval to receive query stats. If not provided the stats information is not sent | optional[int] |
Successful retrieval of system state
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$License successfully updated and current license info returned
It will update the connections state and validate the configuration. If the validation fails, the state will not be updated.
It will only validate the request, not applying any actual change to the system.
It will try to connect to the configured service as part of the validation step.
Configuration in YAML format representing the connections state.
The only allowed name for the Kafka connection is "kafka".
Kafka security protocol.
SSL keystore file path.
Password to the keystore.
Key password for the keystore.
Password to the truststore.
SSL truststore file path.
JAAS Login module configuration for SASL.
Kerberos keytab file path.
Comma separated list of protocol://host:port to use for initial connection to Kafka.
Mechanism to use when authenticated using SASL.
Default port number for metrics connection (JMX and JOLOKIA).
The username for metrics connections.
The password for metrics connections.
Flag to enable SSL for metrics connections.
HTTP URL suffix for Jolokia or AWS metrics.
HTTP Request timeout (ms) for Jolokia or AWS metrics.
Metrics type.
Additional properties for Kafka connection.
Mapping from node URL to metrics URL, allows overriding metrics target on a per-node basis.
DEPRECATED.
The only allowed name for a schema registry connection is "schema-registry".
Path to SSL keystore file
Password to the keystore
Key password for the keystore
Password to the truststore
Path to SSL truststore file
List of schema registry urls
Source for the basic auth credentials
Basic auth user information
Metrics type
Flag to enable SSL for metrics connections
The username for metrics connections
The password for metrics connections
Default port number for metrics connection (JMX and JOLOKIA)
Additional properties for Schema Registry connection
Mapping from node URL to metrics URL, allows overriding metrics target on a per-node basis
DEPRECATED
HTTP URL suffix for Jolokia metrics
HTTP Request timeout (ms) for Jolokia metrics
Username for HTTP Basic Authentication
Password for HTTP Basic Authentication
Enables Schema Registry hard delete
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The username to connect to the Elasticsearch service.
The password to connect to the Elasticsearch service.
The nodes of the Elasticsearch cluster to connect to, e.g. https://hostname:port. Use the tab key to specify multiple nodes.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An Integration Key for PagerDuty's service with Events API v2 integration type.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The Datadog site.
The Datadog API key.
The Datadog application key.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The Slack endpoint to send the alert to.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Comma separated list of Alert Manager endpoints.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The host name.
An optional port number to be appended to the hostname.
Set to true in order to set the URL scheme to https. Will otherwise default to http.
An array of (secret) strings to be passed over to alert channel plugins.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Way to authenticate against AWS.
Access key ID of an AWS IAM account.
Secret access key of an AWS IAM account.
AWS region to connect to. If not provided, this is deferred to client configuration.
Specifies the session token value that is required if you are using temporary security credentials that you retrieved directly from AWS STS operations.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$List of Kafka Connect worker URLs.
Username for HTTP Basic Authentication.
Password for HTTP Basic Authentication.
Flag to enable SSL for metrics connections.
The username for metrics connections.
The password for metrics connections.
Metrics type.
Default port number for metrics connection (JMX and JOLOKIA).
AES256 Key used to encrypt secret properties when deploying Connectors to this ConnectCluster.
Name of the ssl algorithm. If empty default one will be used (X509).
SSL keystore file.
Password to the keystore.
Key password for the keystore.
Password to the truststore.
SSL truststore file.
Mapping from node URL to metrics URL, allows overriding metrics target on a per-node basis.
DEPRECATED.
HTTP URL suffix for Jolokia metrics.
HTTP Request timeout (ms) for Jolokia metrics.
The only allowed name for a schema registry connection is "schema-registry".
Way to authenticate against AWS. The value for this project corresponds to the AWS connection name of the AWS connection that contains the authentication mode.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Access key ID of an AWS IAM account. The value for this project corresponds to the AWS connection name of the AWS connection that contains the access key ID.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Secret access key of an AWS IAM account. The value for this project corresponds to the AWS connection name of the AWS connection that contains the secret access key.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Specifies the session token value that is required if you are using temporary security credentials that you retrieved directly from AWS STS operations.
Enter the Amazon Resource Name (ARN) of the Glue schema registry that you want to connect to.
The period in milliseconds that Lenses will be updating its schema cache from AWS Glue.
The size of the schema cache.
Type of schema registry connection.
Default compatibility mode to use on Schema creation.
The only allowed name for the Zookeeper connection is "zookeeper".
List of zookeeper urls.
Zookeeper /znode path.
Zookeeper connection session timeout.
Zookeeper connection timeout.
Metrics type.
Default port number for metrics connection (JMX and JOLOKIA).
The username for metrics connections.
The password for metrics connections.
Flag to enable SSL for metrics connections.
HTTP URL suffix for Jolokia metrics.
HTTP Request timeout (ms) for Jolokia metrics.
Mapping from node URL to metrics URL, allows overriding metrics target on a per-node basis.
DEPRECATED.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The Postgres hostname.
The port number.
The database to connect to.
The user name.
The password.
The SSL connection mode as detailed in https://jdbc.postgresql.org/documentation/head/ssl-client.html.
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$The host name for the HTTP Event Collector API of the Splunk instance.
The port number for the HTTP Event Collector API of the Splunk instance.
Use SSL.
This is not encouraged but is required for a Splunk Cloud Trial instance.
HTTP event collector authorization token.
The only allowed name for the Zookeeper connection is "kerberos".
Kerberos krb5 config
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$Attached file(s) needed for establishing the connection. The name of each file part is used as a reference in the manifest.
Successfully updated connection state
An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$An alphanumeric or dash non-empty string.
^[a-zA-Z0-9-]+$