Virtual server TCP protocol setting and source address persistence setting

Icreated a standard Virtual server. Using the default TCP profile. idele timeout value is 300 seconds with this I also have configured a source address persistence with the timeout value of 2400 seconds. The issue is, the user loggged out after 200 seconds but the server connection doen't timeout after 300 seconds, the connection stays on for 2400 seconds. My understanding is LTM connection table maintains whcih client hit whcih pool member and within the 2400 seconds if the client returns then send the client to the same pool member. I am lost as to why the server connection not timeing out. Thanks in Advance