Delay/Rate-Limit

the delay/rate limit processor controls the flow of messages through an analytics flow it either delays each message before forwarding it or limits how many messages pass through per time window use it to smooth bursty traffic, pace calls to a downstream system, or throttle a noisy source modes the action parameter selects the behavior delay each message holds each message for a period before forwarding it rate limit caps how many messages pass through in a time window delay mode when action is delay each message , the delay mode parameter sets the strategy fixed delay delays every message by a constant time, set by delay time and delay time units random delay delays each message by a random time between a minimum and a maximum, set by delay random time from and delay random time to and their units get delay from payload reads the delay from a numeric delay field in the message when that field is missing, the processor falls back to delay time rate limit mode when action is rate limit , the rate mode parameter sets the scope all messages applies the limit to all messages together for each payload topic applies the limit per topic, using the topic field in the payload, and releases messages round robin across topics the processor allows rate msg count messages per rate msg time window, measured in rate msg time units enable rate override from payload to take the rate from a rate field in the message instead in rate limit mode, the processor keeps only the latest message and drops the intermediate ones intermediate messages rate intermediate mode controls what happens to messages that exceed the delay or rate limit queue intermediate payloads buffers messages for later delivery, up to 1000 when the queue is full, the processor drops new messages to protect memory drop intermediate payloads discards any message that exceeds the limit parameters note the delay fields appear only when action is delay each message , and the rate fields appear only when action is rate limit the random delay fields appear only when delay mode is random delay parameters details action the mode of operation delay each message or rate limit delay mode the delay strategy fixed delay , random delay , or get delay from payload delay time the delay for fixed delay mode, and the fallback for get delay from payload mode when the delay field is missing default is 1 delay time units the units for the fixed delay milliseconds, seconds, minutes, hours, days, or weeks delay random time from the minimum delay used in random delay mode default is 1 delay random time from units the units for the minimum delay delay random time to the maximum delay used in random delay mode default is 10 delay random time to units the units for the maximum delay rate mode the rate limiting scope all messages or for each payload topic rate msg count the number of messages allowed per interval default is 1 rate msg time the size of the rate limit time window default is 1 rate msg time units the units for the rate interval milliseconds, seconds, minutes, hours, days, or weeks rate override from payload allows the rate to be overridden by a rate field in the payload rate intermediate mode how the processor handles messages that exceed the delay or rate limit queue intermediate payloads or drop intermediate payloads