Forum Discussion

hpwilliams_1786's avatar
hpwilliams_1786
Icon for Nimbostratus rankNimbostratus
Nov 28, 2014

LB_FAILED triggered for one specific client to server connection

Hello,

We're using BIG-IP LTM, and we're having a problem with one particular IP address causing an LB_FAILED when trying to connect to a pool

Here is relevant part of the redirect rule:

when HTTP_REQUEST {
  ...
  elseif { [HTTP::uri] starts_with "/core/" } {
    log local0. "XXX core URL found, sending to my_pool [IP::client_addr]"
    pool my_pool
  }
...

And here is the maintenance rule:

when LB_FAILED {
  LB::detach
  log local0. "XXX LB_FAILED for ip [IP::client_addr]"
  ...
}

And here is the logged output (IP changed to protect privacy - it's an external static IP address). The events are 4 seconds apart.

Rule /redirect : XXX core URL found, sending to my_pool *.*.*.71 
Rule /maintenance-page : XXX LB_FAILED for ip *.*.*.71 

Now, my_pool contains two machines and they're both definitely up. Nothing else is triggering LB_FAILED, only this particular client. Connections from *.72, which is physically next to the *.71 one and with the same ISP work. Changing client machines, browsers, etc. does not work, and the same clients can connect from other IP addresses.

Does anyone know why an LB_FAILED might be triggered in this case? The address is not blacklisted.

Thanks in advance

4 Replies

  • R_Marc's avatar
    R_Marc
    Icon for Nimbostratus rankNimbostratus
    are you SNATing? The only thing that would come to mind is if the you aren't SNATing and the back end has a specific route for the IP in question. I would start with a tcpdump and/or ssldump.
  • We're not NATing or SNATing. I ran TCPDump on a machine in a pool that this IP address can reach and I can see the F5 BIGipServer~ cookies being set for requests from *.71. It seems that it is only my-pool that cannot be reached. Speaking of BIGipServer cookies, if these are being set with the wrong IP address would that cause the problem. I hit the load balancer with curl and that cookie set to a junk value, but I was sent to my_pool with no problem.
  • You can try to map your client ip to a specific pool member, one by one and compare the results. Probably some endpoint protection running on server is rejecting the request.

     

    when HTTP_REQUEST { if { [IP::addr [IP::client_addr] equals ..*.71] } pool my-pool member } }

     

    and then change it to member2_IP.

     

  • nitass/R Marc: We ran tcpdump while the client connected and received nothing from that address, but they still got the maintenance page. This suggests some routing or packet manipulation issue before the packet even gets to F5. Client are now using a VPN which is getting them through.

     

    Pratik: We'll some explicit rules next.

     

    Thanks for everyone's help