Windowing

Windowing allows you to control how to group records that share the same key for stateful operations such as aggregations and window joins. Windows are tracked on a per record key basis. Lenses SQL has support for the full spectrum of the windowing functionality that is available in the Kafka Streams API.

Note

A record is discarded and will not be processed by the window if it arrives after the retention period – that is, if the retention period has passed.

The following types of windows are supported:

  • Hopping time windows. These windows are based on time intervals. They model fixed-sized, (possibly) overlapping windows. A hopping window is defined by two properties: the window size and its advance interval (aka “hop”). The advance interval specifies how much a window moves forward relative to the previous one.

For example, you can configure a hopping window with a size of 5 minutes and an advance interval of 1 minute as follows:

...
GROUP BY HOP(5,m,1,m)
...

Since hopping windows can overlap, a data record may belong to more than one such window.

  • Tumbling time windows. These are a special case of hopping time windows and, like the latter, are windows based on time intervals. They model fixed-size, non-overlapping, gap-less windows. A tumbling window is defined by a single property: the window size. A tumbling window is a hopping window whose window size is equal to its advance interval.
...
GROUP BY tumble(1,m)
...

Since tumbling windows never overlap, a data record will belong to one and only one window.

  • Sliding windows. These express fixed-size window that slides continuously over the time axis. Here, two data records are said to be included in the same window if the difference of their timestamps is within the window size. Thus, sliding windows are not aligned with the epoch, but on the data record timestamps.
...
GROUP BY SLIDING(1,m)
...
  • Session windows. These are used to aggregate key-based events into sessions. Sessions represent a period of activity separated by a defined gap of inactivity. Any events processed that fall within the inactivity gap of any existing sessions are merged into the existing sessions. If the event falls outside of the session gap, then a new session will be created. Session windows are tracked independently across keys (e.g. windows of different keys typically have different start and end times) and their sizes vary (even windows for the same key typically have different sizes). As such session windows can not be precomputed, they are derived from analyzing the timestamps of the data records.
...
GROUP BY SESSION(10,m, 5, m)
...

All the window functions allow the user to specify the time unit. Supported time windows are:

Keyword Unit
MS milliseconds
S seconds
M minutes
H hours