Anomaly Detection
5 min
the anomaly detection processor flags data points that fall outside a normal statistical range using the three sigma rule you define how many standard deviations count as normal for your data set anything beyond that range is flagged as anomalous why use anomaly detection use this processor to catch drift or outliers in a data stream without hand writing threshold logic for every tag, for example, flagging a sensor reading that suddenly spikes outside its normal operating range how it works the processor observes a rolling window of values and continuously calculates their average and standard deviation you choose how many standard deviations from that average count as normal everything beyond it is anomalous when an anomaly occurs, the processor expands the standard deviation window slightly so a single large anomaly doesn't drastically skew the moving average and standard deviation if the anomalous behavior turns out to be seasonal rather than a one off if you don't need this adjustment and only want a simple upper or lower limit check, enable control chart mode? to bypass it with control chart mode enabled, the processor makes no changes to the moving window before calculating the limits very small deviations (close to zero) are harder to distinguish from anomalies if your data has small deviations, use a larger window size the processor outputs the following fields timestamp, current value, moving average, moving standard deviation, upper limit, lower limit, and total anomalies if the current value is flagged as anomalous, an anomaly field replaces it parameters parameter details control chart mode? bypasses the standard deviation window adjustment and performs a simple upper or lower limit check instead deviations controls how strict the "normal" range is values within n standard deviations of the average are normal; values outside it are flagged window size the rolling window (in seconds) over which the average and standard deviation are calculated pass through value determines whether the processor's output is the anomaly detection results or the original unchanged input value standard deviation is a measure of spread low means values are close together, and high means they're more spread out this uses bessel's correction (`n 1`), so the input must contain at least two values for example, `\[1, 2, 3, 4]` produces a mean of `2 5` and a standard deviation of approximately `1 291` common choices are `2` (stricter) or `3` (more forgiving) \</td> limitations and considerations very small deviations (close to zero) are difficult to distinguish from genuine anomalies use a larger window size for this kind of data related topics use the anomaly detection function docid\ ci 9aglo2iwr2ot7v0tvf