Clear the active connections.

They should clear gradually over time. You could force all connections to move by removing the Pool Member from the Pool entirely but this could be disruptive. In the same vein I think changing the Action on Service Down from None to Reselect may help.

I'm not sure it's an option with your setup but OneConnect typically helps speed up the 'draining' of connections from a disabled Pool Member. You could also reduce the TCP idle timeout of the VS itself but this may have negative affects.