Load balancing not working

I have 2 Real servers and I am using default source address persistence. As one my real server goes down, the connection does not move to other load balancer. Where could b the issue?