Forum Discussion

Mark_Melin_6298's avatar
Mark_Melin_6298
Icon for Nimbostratus rankNimbostratus
May 13, 2008

Complex Session Matching...

Im trying to design some large site infrastructure where i have two clusters

 

 

cluster1 is public and serves hundreds of thousands of persistent tcp connections for registration

 

cluster2 is private and brokers instructions from another pool of servers into first pool

 

 

goal is on connect to cluster2, query the session table for the cluster1 pool member

 

and subsquently send the request for cluster2 to the same pool member for the session

 

queried in cluster1

7 Replies

  • The "Match Across Virtuals" checkbox in the persistence profile will do what you are asking when the connections come to cluster1 and cluster2 from same client, but I'm not sure that's what you have in mind.

     

     

    Denny
  • that's half of what i need... i need to match client2->virtual2 to same pool member as client1->virtual1 based on a known persistence string... like a predefined string in an xml or cookie for example
  • i've queried my F5 sales engineering folks on this thread, as a followup, assuming we can 'match the session' as described above... now we want to solve for what to match on... the question is:

     

     

    'If our protocol is XML, can we pass it to our client via straight

     

    TCP and have the F5 parse out "cookie" info? or do we need to

     

    use HTTP to transport the file (ie have an HTTP header that

     

    specifies cookie info)?'
  • You can collect the TCP data and parse out any strings you might need to. From an iRule perspective, it would be easier to use HTTP to send the data and be able to use HTTP::cookie to parse the cookie info.

     

     

    Aaron
  • I solved this for the most part, and have successfully loaded this with 400K connected clients and over 1000 concurrent messages delivered to those clients.... as follows:

    iRules Procedure:

    - Collect TCP Payload, Inspect TCP Payload for String

    - MD5 hashing string TCP payload concatenated with each node ip address in foreach loop

    - binary scan the MD5 hash value for each node and return the first byte's integer value

    - compare each score and send the connecction to the winning node

    - repeat the process for the second virtual server, the same node is selected for both connections

    Handling Node Failure:

    - create custom monitor for each service, set as monitor for pools

    - set 'manual resume' to yes in both monitors, avoids assymetric conns if host is restored without intervention

    - configure pools for 'action on service down' as 'reject', sends RST to all clients on that node, they reconnect, re-calc and relect new winner

    - configure tcp profiles for 'infinite' keepalive interval' which disables keepalive ACKs from lb to clients

    - configure same tcp profiles for 'zero time wait' 'zero fin wait' and 'zero close wait' and low 'idle timeout'

    - clients elect a new node on fail, and timeout after specified interval thus re-electing on specified interval

    Caveats:

    - communications are assymetrical, and thus broken, for the specified interval above, when a node in added to pool

    - sending RST for 1M x 1/nodes is less than optimal, would like to send RSTs on a known failure rather than simply status change

    Under Consideration:

    - build a secondary vs for msging which has exact same characteristics as vs's mentioned above

    - do not restore the pool member on the second vs yet

    - apply same iRule that scores the winner, but this time it picks the second server (the winner before adding new member)

    - have the sending msg server 'retry' to the second vip, the message should succeed, THEN

    - application logic to have the msg sender send a reconnect instruction to client app

    -OR-

    - create persistence record on the TCP payload string above AND

    - iRule that queries persistence table, returns foreign ip and port, sends TCP RST to client

    - client reconnects and elects correct winner

    iRule 1 - Client Rule for MD5 Hash Node Election

    Loadbalancing Based on TCP Payload String, Node IP and MD5 Hash Score

        
         when CLIENT_ACCEPTED {     
           TCP::collect 2     
         }     
         when CLIENT_DATA {     
           set msg [findstr [TCP::payload] "" 0 "\r\n"]     
           set sid [findstr $msg "" 9 "\r\n"]     
          set S ""      
          foreach N [active_members -list myClientPool] {      
            set I [lindex $N 0]     
            binary scan [md5 $I$sid] c1 NMD5     
            if { $NMD5 > $S } {     
              set S $NMD5     
              set W $N     
            } else {     
                  set S $NMD5     
            }     
          }      
           pool myClientPool member [lindex $W 0] [lindex $W 1]     
         }     
         

    iRule 2 - Msg Server Rule for MD5 Hash Node Election

    Loadbalancing Based on TCP Payload String, Node IP and MD5 Hash Score

         
         when CLIENT_ACCEPTED {     
           TCP::collect 2     
         }     
         when CLIENT_DATA {     
           set msg [findstr [TCP::payload] "" 0 "\r\n"]     
           set sid [findstr $msg "" 9 "\r\n"]     
          set S ""      
          foreach N [active_members -list myMsgSrvPool] {      
            set I [lindex $N 0]     
            binary scan [md5 $I$sid] c1 NMD5     
            if { $NMD5 > $S } {     
              set S $NMD5     
              set W $N     
            } else {     
                  set S $NMD5     
            }     
          }      
           pool myMsgSrvPool member [lindex $W 0] [lindex $W 1]     
         }     
         
  • please note any rule that uses "active_members -list " requires version 9.4.2+

     

     

    CB
  • Deb_Allen_18's avatar
    Deb_Allen_18
    Historic F5 Account
    Excellent use of the election hash algorithm. This is exactly the kind of problem that is intended to solve - cases where the answer must be determined with no historical data or awareness of assessment made by another device.

     

     

    Please consider adding them to the codeshare (Click here)

     

     

    /d