Azure KeyVault

This page describes how to retrieve secrets from Azure KeyVault for use in Kafka Connect.

Secure secrets in Azure KeyVault and use them in Kafka Connect.

Secrets will only be reloaded if the Connector restarts.

Authentication

Two authentication methods are supported:

1. credentials. When using this configuration the client-id, tenant-id and secret-id for an Azure service principal that has access to key vaults must be provided

2. default. This method uses the default credential provider chain from Azure. The default credential first checks environment variables for configuration. If the environment configuration is incomplete, it will try to use managed identities.

Configuring the plugin

Name	Description	Default
azure.auth.method	Azure authenticate method. ‘credentials’ to use the provided credentials or ‘default’ for the standard Azure provider chain	credentials
azure.client.id	Azure client id for the service principal. Valid is auth.method is ‘credentials’
azure.tenant.id	Azure tenant id for the service principal. Valid is auth.method is ‘credentials’
azure.secret.id	Azure secret id for the service principal. Valid is auth.method is ‘credentials’
file.dir	The base location for any files to stored

Example worker properties file:

worker.props

config.providers=azure
config.providers.azure.class=io.lenses.connect.secrets.providers.AzureSecretProvider
config.providers.azure.param.azure.auth.method=credentials
config.providers.azure.param.azure.client.id=your-client-id
config.providers.azure.param.azure.secret.id=your-secret-id
config.providers.azure.param.azure.tenant.id=your-tenant-id
config.providers.azure.param.file.dir=/connector-files/azure

Usage

To use this provider in a connector, reference the keyvault containing the secret and the key name for the value of the connector property.

The indirect reference is in the form ${provider:path:key} where:

• provider is the name of the provider in the worker property file set above

• path is the URL of the Azure KeyVault. DO NOT provide the https:// protocol for the in the keyvault name as the Connect worker will not parse it correctly

• key is the name of the secret key in the Azure KeyVault

For example, if we store two secrets:

• my_username with the value lenses and

• my_password with the value my-secret-password

in a Keyvault called my-azure-key-vault we would set:

connector.props

name=my-sink
class=my-class
topics=mytopic
username=${azure:my-azure-key-vault.vault.azure.net:my_username}
password=${azure:my-azure-key-vault.vault.azure.net:my_password}

This would resolve at runtime to:

name=my-sink
class=my-class
topics=mytopic
username=lenses
password=my-secret-password

Data encoding

The provider handles the following types:

• utf_8

• base64

The provider will look for a tag attached to the secret called file-encoding. The value for this tag can be:

• UTF8

• UTF_FILE

• BASE64

• BASE64_FILE

The UTF8 means the value returned is the string retrieved for the secret key. The BASE64 means the value returned is the base64 decoded string retrieved for the secret key.

If the value for the tag is UTF8_FILE the string contents as are written to a file. The returned value from the connector configuration key will be the location of the file. The file location is determined by the file.dir configuration option is given to the provider via the Connect worker.properties file.

If the value for the tag is BASE64_FILE the string contents are based64 decoded and are written to a file. The returned value from the connector configuration key will be the location of the file. For example, if a connector needs a PEM file on disk, set the prefix as BASE64_FILE. The file location is determined by the file.dir configuration option is given to the provider via the Connect worker.properties file.

If no tag is found the contents of the secret string are returned.