# Plugins

The following implementations can be specified:

1. **Serializers/Deserializers** Plug your serializer and deserializer to enable observability over any data format (i.e., protobuf / thrift)
2. **Custom authentication** Authenticate users on your proxy and inject permissions HTTP headers.
3. **LDAP lookup** Use multiple LDAP servers or your group mapping logic.
4. **SQL UDFs** User Defined Functions (UDF) that extend SQL and streaming SQL capabilities.

Once built, the jar files and any plugin dependencies should be added to the Agent and, in the case of Serializers and UDFs, to the SQL Processors if required.

## Adding plugins <a href="#adding-plugins" id="adding-plugins"></a>

On startup, the Agent loads plugins from the **$LENSES\_HOME/plugins/** directory and any location set in the environment variable **LENSES\_PLUGINS\_CLASSPATH\_OPTS**. The Agent is watching, and dropping a new plugin will hot-reload it. For the Agent docker (and Helm chart) you use **/data/plugins.**

Any first-level directories under the paths mentioned above, detected on startup will also be monitored for new files. During startup, the list of monitored locations will be shown in the logs to help confirm the setup.

```bash
...
Initializing (pre-run) Lenses
Installation directory autodetected: /opt/lenses
Current directory: /data
Logback configuration file autodetected: logback.xml
These directories will be monitored for new jar files:
 - /opt/lenses/plugins
 - /data/plugins
 - /opt/lenses/serde
Starting application
...
```

Whilst all jar files may be added to the same directory (e.g **/data/plugins**), it is suggested to use a directory hierarchy to make management and maintenance easier.

An example hierarchy for a set of plugins:

```bash
├── security
│   └── sso_header_decoder.jar
├── serde
│   ├── protobuf_actions.jar
│   └── protobuf_clients.jar
└── udf
    ├── eu_vat.jar
    ├── reverse_geocode.jar
    └── summer_sale_discount.jar
```

## SQL Processors in Kubernetes <a href="#sql-processors-in-kubernetes" id="sql-processors-in-kubernetes"></a>

There are two ways to add custom plugins (UDFs and Serializers) to the SQL Processors; (1) via making available a *tar.gz* archive at an HTTP (s) address, or (2) via creating a custom docker image.

## **Archive served via HTTP**

With this method, a tar archive, compressed with gzip, can be created that contains all plugin jars and their dependencies. Then this archive should be uploaded to a web server that the SQL Processors containers can access, and its address should be set with the option **lenses.kubernetes.processor.extra.jars.url**.

Step by step:

1. Create a *tar.gz* file that includes all required jars at its root:

   ```bash
   tar -czf [FILENAME.tar.gz] -C /path/to/jars/ *
   ```
2. Upload to a web server, ie. `https://example.net/myfiles/FILENAME.tar.gz`
3. Set

   ```bash
   lenses.kubernetes.processor.extra.jars.url=https://example.net/myfiles/FILENAME.tar.gz
   ```

   For the docker image, set the corresponding environment variable

   ```bash
   LENSES_KUBERNETES_PROCESSOR_EXTRA_JARS_URL=https://example.net/myfiles/FILENAME.tar.gz`
   ```

## **Custom Docker image**

The SQL Processors inside Kubernetes use the docker image **lensesio-extra/sql-processor**. It is possible to build a custom image and add all the required jar files under the **/plugins** directory, then set **lenses.kubernetes.processor.image.name** and **lenses.kubernetes.processor.image.tag** options to point to the custom image.

Step by step:

1. Create a Docker image using **lensesio-extra/sql-processor:VERSION** as a base and add all required jar files under **/plugins**:

   ```bash
   FROM lensesio-extra/sql-processor:4.2
   ADD jars/* /plugins
   ```

   ```bash
   docker build -t example/sql-processor:4.2 .
   ```
2. Upload the docker image to a registry:

   ```bash
   docker push example/sql-processor:4.2
   ```
3. Set

   ```bash
   lenses.kubernetes.processor.image.name=example/sql-processor
   lenses.kubernetes.processor.image.tag=4.2
   ```

   For the docker image, set the corresponding environment variables

   ```bash
   LENSES_KUBERNETES_PROCESSOR_IMAGE_NAME=example/sql-processor
   LENSES_KUBERNETES_PROCESSOR_IMAGE_TAG=4.2
   ```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.lenses.io/latest/deployment/configuration/agent/plugins.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.