Unwrapping complex types
This page describes a tutorial to unwrap a complex data type in a Kafka topic using Lenses SQL Processors.
Last updated
This page describes a tutorial to unwrap a complex data type in a Kafka topic using Lenses SQL Processors.
Last updated
2024 © Lenses.io Ltd. Apache, Apache Kafka, Kafka and associated open source project names are trademarks of the Apache Software Foundation.
In this example, we will show how Lenses can be used to transform complex data types into simple primitive ones.
We start this tutorial by creating a topic which will hold information regarding visits to our website:
Firstly we’ll add some data to our newly created topic:
For example, let’s say we’re interested in sending this data to a service that analyses the time spent on a page and how it changes over time.
This system has a caveat though it only accepts data where keys are specified as strings and values are specified as integers.
Rather than having to reimplement our analysis system, we can create a SQL Processor that will continuously send data to a new topic in a format the target system can work with:
Notice the addition of the as _key
and as _value
aliases; these tell lenses to “unwrap” the values; effectively making lenses write them as primitive types (string and integer respectively) instead of (in this particular case) Avro objects.
Lenses will also automatically infer the format of each topic facet, in this case it set them to STRING and INT respectively.
--