iRule rate limiting with queuing of requests

Hi Nigel88,

your requirement was somewhat interesting for me - so I decided to give it a try... 😉

The iRule below is my interpretation of your requirement. The iRule is implementing a Sliding-Window based request limiter with configurable max-rate and interval settings. Once the configured request/sec value has been reached, a second Sliding-Window limiter is executed to maintain the required request queue. If a free slot in the request queue is available, the ongoing HTTP request will be paused for a configurable retry interval. After the retry interval has been elapsed the current request/sec limiter value will be evaluated again and depending on the outcome the ongoing HTTP request will be released from the queue or paused for another retry interval (till the queue timeout has been reached). If the queue slots are exhausted or if the queue timeout has been reached a HTTP-Error Code 429 will be send to the client.

Let me know if this fits your requirement...

Note: The implemented queue is not FIFO (first in first out) based. So the queued requests may be released/droped in an unsorted manner. But it should be sufficient to catch certain bursty traffic spikes...

when RULE_INIT {

     Tweak the request limit and interval 
    set static::request_limit 14                ;count
    set static::request_interval 1              ;sec

     Tweak the queue size, timeout and retry interval
    set static::queue_size 250                  ;count
    set static::queue_timeout 5                 ;sec
    set static::queue_retry_interval 1000       ;msec
}

when HTTP_REQUEST {

     Enforce limits only on GET requests...

    if { [HTTP::method] eq "GET" } then {

         Calculate a unique request ID to track the request...
        set request_id "[TMM::cmp_unit][clock clicks]"

         Compute table labels for request and queued limiter...
        set request_label "R_[HTTP::host]:[IP::client_addr]"
        set queue_label "Q_[HTTP::host]"

         Checking if request limit has been reached...

        if { [set request_rate [table keys -count -subtable $request_label]] < $static::request_limit } then {

             Max request rate is not exceeded. Insert a new entry to request limiter table...
            table set -subtable $request_label $request_id "1" indef $static::request_interval 
            log local0.debug "Debug: $request_id : Allow: Request rate for $request_label is $request_rate / $static::request_interval seconds"
             Allowing the request to pass...

        } else {

             Max request rate is exceeded. Holding the request to check queue slots...
            log local0.debug "Debug: $request_id : Hold: Request rate for $request_label is $request_rate / $static::request_interval seconds"

             Checking if queue has a free slot....
            if { [set queue_size [table keys -count -subtable $queue_label]] < $static::queue_size } then {

                 Queue is not exceeded. Insert a new entry to queue limiter table...
                table set -subtable $queue_label $request_id "1" indef $static::queue_timeout 
                log local0.debug "Debug: $request_id : Queued: Queue counter for $queue_label is $queue_size"

                 Initialize queue timer...
                set queue_time 0

                 Enforcing the queue loop timeout timer...
                while { $static::queue_timeout * 1000 > $queue_time } {

                     Pausing the ongoing HTTP request for queue retry interval
                    after $static::queue_retry_interval

                     Checking if request limit has been lowered...
                    if { [set request_rate [table keys -count -subtable $request_label]] < $static::request_limit } then {

                         Max request rate has been lowered. Insert a new entry to our request rate limiter...
                        table set -subtable $request_label $request_id "1" indef $static::request_interval 
                        log local0.debug "Debug: $request_id : Queue Release: Requests rate for $request_label is now $request_rate / $static::request_interval seconds"

                         Removing this request from request queue
                        table delete -subtable $queue_label $request_id

                         Exiting this iRule and allowing the request to pass...
                        return

                    } else {

                         Max request rate is still exceeded. Queueing the request another cycle...

                        log local0.debug "Debug: $request_id : Queue Retry: Requests rate for $request_label is now $request_rate / $static::request_interval seconds"
                        incr queue_time $static::queue_retry_interval

                    }
                }

                 Queued request has been timed out. Dropping the request...
                log local0.debug "Debug: $request_id : Queue Fail: Requests rate for $request_label is now $request_rate / $static::request_interval seconds"
                HTTP::respond 429 content "Request Denied: Exceeded requests/sec limits"

            } else {

                 Maximum queue size has been exceeded. Dropping the request...
                log local0.debug "Debug: $request_id : Queue Exceeded: Queue counter for $queue_label is $queue_size"
                HTTP::respond 429 content "Request Denied: Exceeded requests/sec limits"

            }
        }
    }
}

Cheers, Kai