NimbostratusIs there a default setting for HTTP retries? We are coming from Nginx loadbalancers, which retried once per server if a request failed (very bad default, I know). I've found documentation for an HTTP:retry here, and I can easily set a rule to retry on requests that respond with 5xx errors. However, I just wanted to know if by default LTM retries HTTP requests with 5xx errors before I add my own layering on top.
EmployeeI believe there is no setting to configure an automatic retry on error responses.
NimbostratusThat's what I thought as well. It seems like there is a way to do a http retry on error codes, but is there a way to do it on connection refused? The issue we're having is that a pool member goes down, but it's still receiving requests before the health monitor marks it as down. All those requests will be getting a connection refused error, is there a way to retry those requests on other working pool members?
EmployeeCertainly, and the first example on the HTTP::retry wiki page is probably pretty close to what (I think) you're looking for. On HTTP response, if the server sends back a status code that indicates an error, you retry the entire request and select a new pool member.
NimbostratusI understand that, but the server isn't sending back a status code. If we turn off apache, nothing is being sent. The F5 should just be receiving a connection refused error, not accompanied by an HTTP error code, because nothing will be listening on port 80.
I ask this because we're trying to figure out how to handle out rolling code to our servers. We do a rolling restart, which means that each server is down for 2-4 seconds, not long enough to be marked as down by a health check, but all requests in that 2-4 seconds are being blackholed. I'd like the F5 to resend those requests to other working pool members when it sees that the connection from him to the pool member is not working.
EmployeeThe F5 should just be receiving a connection refused error, not accompanied by an HTTP error code, because nothing will be listening on port 80
If you turned off the port 80 listener, would you get any response at all? I'm thinking the TCP handshake would never complete. Please do a tcpdump and confirm if you're seeing an actual response from the server, or the BIG-IP just repeatedly sending an unanswered SYN packet. If the latter is the case, you would probably need a fairly aggressive TCP monitor applied to the pool as well.
NimbostratusSo we figured out how to fix this, and it's actually a very simple fix. In the pool's settings, go to the advanced menu, and set the Reselect Tries value to something other than 0. This will reselect a pool member for the request on a passive failure (i.e. connection refused) and retry the request. Our testing show that this works perfectly for us and solves our issue.
CirrostratusYou've already solved the issue, but here's a relevant link explaining the behavior:
http://support.f5.com/kb/en-us/solutions/public/10000/600/sol10640.html?sr=32321461