Time Span Filtering

Technical Complexity

Medium

Filtering Class

Information Aggregation

Purpose

Avoid false detection of transient faults. These faults can be logged and monitored by another instance, but this end point will only receive stable faults.
Wait a certain period of time, and forward only the dominant event within that period to the event listener.

Situation

A network connection is deemed unpredictable. It has fluke (transients) failures occurring all the time. But only when the network will be down more than 20% of the time, that the fault will be send to the event listener.

Design

IEP deals with streams, but allow us to work with relations with this the process. In this case, we want to do the following:

Set a time window that is large enough to avoid boundary conditions, but small enough to make quick decisions
Count the number of events that will allow us to make a decision on the outcome (ON/OFF events)
Make the decision which is the most relevant event.
Each time a decision is made (every 10 seconds), and that a higher than 20% down rate is detected, insert a new event it the outbound stream.

IEP Constructs Used

Process Flow

Inbound stream, System State Event Stream
Time Based Aggregation Window Converter
On and Off counts
Relation joint, decision point
Convert to insert event for the output stream
Output stream, Fluke count stream

Time Based Aggregation

In this sample, we will do an event aggregation every 10 seconds, which will take into account all events that occured in the last 30 seconds. This provides us with a "moving average" model we can use here.

Relevant Event Type Counts

On Counts

Off Counts

Here, we count the number of event we want from the time based aggregation. This allow us to create a stream of a single event type, so we can process it to generate a single summary event that is relevant to us.

Decision point, the ON/OFF count comparison

At the decision point, we are left with the simple need to compare which count we have the most and to forward this event. Only when the OFF rate is greater than a 5th of the ON rate, an event will be forwarded to the outstream.

Sample Files

Time Span Filtering