Streaming System: Advanced Windowing

When/Where: Processing-Time Windows

Processing-Time windows are important for two reasons:

1. For certain usage like usage monitoring. Processing time windows are effective
2. Processing time can help analyze system status
   There are two methods to achieve processing-time windows:
3. Trigger: ignore event time and use trigger to provide windows
4. Ingress time: Assign ingress times as the event times for data as they arrive and use normal event-time windowing from there on.
   The first way may cause different time for one data in different stage. The second method may be better.

Processing-time windows are sensitive to input order and time the data input.

Session Windowing

Sessions are a special type of window that captures a period of activity in the data that is terminated by a gap of inactivity.

Sessions are particularly interesting in two ways:

1. A data-driven windows
2. An example of un-aligned window

The key insight in providing general session support is that a complete session window is, by definition, a composition of a set of smaller, overlapping windows, each containing a single record, with each record in the sequence separated from the next by a gap of inactivity no larger than a predefined timeout.

Where: Custom Windowing
A custom windowing consists of two things:

1. Window alignment: Place each data into an initial window
2. Window merging (optional): one example is session window which will be merged.

One example is unaligned windowing for different key aggregation. If system has thousands of distinct keys to aggregation. Then aligned windowing will cause too many small pieces to simultaneously updating. Custom unaligned windows by key id can mitigate this trouble.