Problem with persistence

Ok, this is *really* weird.

I modified the iRule to log what the current node in the persistence table is just before it does the `persist uie $uid` call. I also ssh'ed into the machine so I could get dumps of the persistence table. I then ran a few test connections with various uid values and between each I looked at the logs and the persistence table. I waited at least 10 seconds between each request so I could tell if the uid persistence timeout was being reset (indicating that that persistence was used). I generated a file that shows the logs for the request (there are 2 lines per request), then the results of `bigpipe persist show all` on the machine. I added the results of this below (edited a bit so the lines are so long). Here's the rule I used:

when HTTP_REQUEST {
  set fulluri [HTTP::uri]
  set uid [findstr [HTTP::uri] "uid=" 4 "&"]
  if { $uid != "" } {
    log local0. "Set persist value to uid: $uid  | $fulluri"
    set currentPersist [persist lookup uie $uid node]
    log local0. "Current persist for $uid is $currentPersist"
    persist uie $uid
  }
  else {
    log local0. "No uid found in HTTP REQUEST  | $fulluri"
  }
}

What I found was that the first request and UID worked fine. The first request showed that there wasn't a value for the uid, and subsequent requests showed that there was. This is correct.

Weird things started happening when I changed the uid value. All the later requests showed logging that it was finding and calling `persist uie $uid` with the correct values. However, the persistence record wasn't being added to the list, and the timeouts on the *old* uid was was being reset, suggesting it was using that persistence record instead. This continued to happen, even if I changed the uid value again.

The only time it correctly used the new uid value seemed to be when I waited long enough for the old one to timeout and be removed (this is what happened with the last test).

Does anyone have any ideas what could be causing this? At this point it looks like a problem with how BIG-IP is handling the call to `persist uie $uid`, as if it's ignoring the value I'm giving it.

PERSISTENT CONNECTIONS --
No persistent connections were found.
Jun 22 15:31:34 : Set persist value to uid: 1  | /testapp?uid=1&sc=0
Jun 22 15:31:34 : Current persist for 1 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  2sec
Jun 22 15:31:41 : Set persist value to uid: 1  | /testapp?uid=1&sc=0
Jun 22 15:31:41 : Current persist for 1 is 10.10.70.32
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  5sec
Jun 22 15:31:50 : Set persist value to uid: 1  | /testapp?uid=1&sc=0
Jun 22 15:31:50 : Current persist for 1 is 10.10.70.32
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  1sec
Jun 22 15:32:03 : Set persist value to uid: 432  | /testapp?uid=432&sc=0
Jun 22 15:32:03 : Current persist for 432 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  15sec
Jun 22 15:32:10 : Set persist value to uid: 432  | /testapp?uid=432&sc=0
Jun 22 15:32:10 : Current persist for 432 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  9sec
Jun 22 15:32:17 : Set persist value to uid: 432  | /testapp?uid=432&sc=0
Jun 22 15:32:17 : Current persist for 432 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  2sec
Jun 22 15:32:28 : Set persist value to uid: 321  | /testapp?uid=321&sc=0
Jun 22 15:32:28 : Current persist for 321 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  14sec
Jun 22 15:32:34 : Set persist value to uid: 321  | /testapp?uid=321&sc=0
Jun 22 15:32:34 : Current persist for 321 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 1
    Virtual: 10.10.70.171:http    Node: 10.10.70.32:http    Age:  7sec
Jun 22 15:35:38 : Set persist value to uid: 90  | /testapp?uid=90&sc=0
Jun 22 15:35:38 : Current persist for 90 is
PERSISTENT CONNECTIONS --
    Mode: universal   Value: 90
    Virtual: 10.10.70.171:http    Node: 10.10.70.39:http    Age:  3sec

Sheesh, I go on vacation one day and no one can seem to answer this for you...

 

 

The problem is that the persist command does not change the already connected back-end server on a subsequent request over the same connection. In order for that to happen, you could either enable OneConnect (add the oneconnect profile to your virtual) or modify the rule to call LB::detach if the uid value changes. Then the subsequent request would utilize the new persist setting to (re)connect to the appropriate server.

 

 

Also, just to be clear about the "persist add" command. In this case it is certainly not needed. That is because the "persist uie" command will add a record if it does not exist. Normally, the "persist add" command is only necessary if the first server response contains the information necessary to persist on (EG, as is the case with JSESSION type persistence).

 

 

Anyway, hope this helps and it's not too late.

Oops, minor bug.

In the case where no UID was given but it wasn't a static page (which is really an error, but needs to be handled gracefully) add:

set persistVal "NOUID"