Forum Discussion

bdavis's avatar
bdavis
Icon for Nimbostratus rankNimbostratus
Jan 29, 2013

Irule Help persistence

I currently wrote the below Irule to service requests coming into a VIP based off of HTTP::host and connection limits on the designated pools. Basically if the user requests www.test.gov then it checks to make sure that there are available members in the pool based off connection limits, if there is then he sends him there if not he gets a "Sorry Page". Same thing goes for the rest of the traffic that doesn't match www.test.gov, it checks to see if the default pool have availability.

 

My question is when connections start exceeding the limits and the "We're Sorry Page" starts getting displayed. It works as intented, however since the connections are based off of tcp connections, the user1 may get in the first time and get a page, but every other GET request he makes may not come back because it's over the limit again. How could I other buid persistence into the Irule or use persistence with my Irule so that say you have 100 connection limit, that the first 100 users sessions stay connected until they are complete. Instead of 200 users fighting over a 100 connections and they are all experiencing intermittency? Also if there are any pointers for cleaning this script up please let me know. I had to write this last night in the middle of a huge outage.

 

10 Replies

  • OK, the rule can be improved for sure but let's deal with your main issue first. I assume you have persistence enabled yes? If so I would recommend you configure the option in the persistence profile to Override Connection Limits. This will keep existing users connected and new connections will get the sorry page. Of course, you'll need to test and perhaps lower to configured connection limit to take account of the override but that shouldn't be too hard. I'll post back shortly regarding the iRule.
  • bdavis's avatar
    bdavis
    Icon for Nimbostratus rankNimbostratus

    Thank you for your help. I have cookie persistence enabled with override connection limit enabled in the profile, however when I tested I would receive the web page content and then my next click on the site I would receive "sorry page". Again thank you for your help.

     

  • Hmmm. I imagine the iRule commands are overriding the normal operation. What we probably need to do is check whether the connection has a persistence entry and bypass the rest of the rule. Sorry, no time to do this now but should get back to you tomorrow.
  • bdavis's avatar
    bdavis
    Icon for Nimbostratus rankNimbostratus
    Cookie persistence is what we are currently utilizing.

     

     

    Also they wanted me to include the ability to load balance RFC 1918 source addresses to a seperate pool in the IRULE not requiring connection limits or anything else in the irule. Basically sending RFC1918 requests to a seperate pool that does not have connection limits applied. What I tried doing is add this to the Irule above the rest of the irule. However when I applied it RFC 1918 addresses where not being sent to seperate pool.

     

     

    when HTTP_REQUEST {

     

    if { [matchclass [IP::remote_addr] equals $::private_net] } {

     

    pool pl_x.x.x.x_http_internal

     

    }

     

     

    With a Data Group List below.

     

     

    private_net

     

     

    10.0.0.0/255.0.0.0

     

    172.16.0.0/255.240.0.0

     

    192.168.0.0/255.255.0.0

     

     

    I have implemented blocks before on VIP for single addresses. But never tried full networks. Is this the issue I'm running into because it's not matching the exact value in the datagroup. opposed to seeing if the addresses falls inside one of these 3 networks? Thank you for your help.
  • Try this instead, also, are you sure the DG is configured to contain addresses?

    
     if { [class match [IP::client_addr] equals private_net] } {
    
  • bdavis's avatar
    bdavis
    Icon for Nimbostratus rankNimbostratus
    Thank you. The class match works. I also got the matchclass to work before reading this post it was because of the _ when i got rid of that symbol it worked. Now if only I could bypass the rest of the irule if a persistence existed.
  • bdavis's avatar
    bdavis
    Icon for Nimbostratus rankNimbostratus
    Also to note I was able to get the persistence to work. However what I run into is, let's say that I have a limit restriction of 100 on each one of my members in the perspective pool. When a user comes in and he is subject to the restriction until he gets a persistence record. Upon being persistent, he then can bypass the connection limit. What the problem is if this happens then your connection limit really isn't doing any good because the persistent records continue to grow, which makes the connections grow on the backend servers. What I really need is a way to control the amount of persistent records that can be issued out at once. That way I can truly control how many users can access the system at a given point.
  • Hmmm. Bit of a tough rock and a hard place situation really. The only thing I can think of is to configure a Expiration for the persistence cookie, however it's still going to be a juggling act lowering the limit to account for the persistence.

     

     

    Also, you should disable persistence when serving the sorry page so a user doesn't get the sorry page but also a persistence cookie (which might be the case now).
  • bdavis's avatar
    bdavis
    Icon for Nimbostratus rankNimbostratus

    Ok. So I kinda stepped back and re thought how to approach this and this is what I have came up with. Instead of usinig connection limits, I'm using session limits. Please take a look at this Irule and tell me if you see any issues or ways to optimize this. Also I don't believe it's possible, but is there anyway to do this with a session based cookie intead of using a timout value?

    I have briefly tested this and it seems to work. I would love to hear how I can improve this and or if anyone sees any errors? Also if it's possible to utilize "session Cookies"?

    when RULE_INIT {
       set static::max_active_clients_1 1500
       set static::max_active_clients_2 100
       set static::sessionCookiePrefix "session"
       set static::sessionTimeout 300
       set static::sorrypage {
            
                
            We're Sorry
                
            
                We're Sorry.  Our servers are currently over capacity and certain areas of our site may be temporarily unavailable.  We're working to resolve the issue as quickly as possible.
            
            
        }
    }
    when CLIENT_ACCEPTED {
      if { ![matchclass [IP::remote_addr] equals $::privatenet] } {
            drop
      }
    }
    when HTTP_REQUEST {
        set my_host_header [HTTP::host]
        if { [matchclass [IP::remote_addr] equals $::privatenet] } {
            pool pl_x.x.x.x_http_internal
        }
        switch -glob [string tolower [HTTP::host]] {
            "www.test1.gov*" {
                set subtableName "sessionLimit-[virtual name]"
                set sessionCookieName1 "$static::sessionCookiePrefix-TD-[virtual name]"
                set need_cookie 0
                if {[HTTP::cookie exists $sessionCookieName1]} {
                    set client_id [HTTP::cookie $sessionCookieName1]
                    set sessiondata [table lookup -subtable $subtableName $client_id]
                    if { $sessiondata != "" } {
                    return
                    }
                }
                set sessionCount [table keys -subtable $subtableName -count]
                 if {$sessionCount < $static::max_active_clients_1} {
                    set need_cookie 1
                    set client_id [format "%08d" [expr { int(100000000 * rand()) }]]
                    set sessionValue [IP::client_addr]
                     table add -subtable $subtableName $client_id $sessionValue $static::sessionTimeout
                    log local0. "New Session ($client_id) added value $sessionValue Timeout $static::sessionTimeout"
                } else {
                    HTTP::respond 200 content $static::sorrypage
                }
               
            }
            "www.test2.gov*" {
                set subtableName "sessionLimit-[virtual name]"
                set sessionCookieName2 "$static::sessionCookiePrefix-2-[virtual name]"
                set need_cookie 0
                if {[HTTP::cookie exists $sessionCookieName2]} {
                    set client_id [HTTP::cookie $sessionCookieName2]
                    set sessiondata [table lookup -subtable $subtableName $client_id]
                    if { $sessiondata != "" } {
                    return
                    }
                }
                set sessionCount [table keys -subtable $subtableName -count]
                if {$sessionCount < $static::max_active_clients_2} {
                    set need_cookie 1
                    set client_id [format "%08d" [expr { int(100000000 * rand()) }]]
                    set sessionValue [IP::client_addr]
                    table add -subtable $subtableName $client_id $sessionValue $static::sessionTimeout
                } else {
                    HTTP::respond 200 content $static::sorrypage
                }
            }
        }
    }
     when HTTP_RESPONSE {
        switch -glob [string tolower $my_host_header] {
            "www.test1.gov*" {
                if {$need_cookie == 1} {
                HTTP::cookie insert name $sessionCookieName1 value $client_id path "/"
                }
            }
            "www.test2.gov*" {
                if {$need_cookie == 1} {
                HTTP::cookie insert name $sessionCookieName2 value $client_id path "/"
                }
            }
        }
    }