Big IP LTM 11.5.1 integration with Cisco ACS 5.3

my knowledge of ACS is limited, but i would expect that in principle an error like this is caused because you hit the wrong rule / policy / ... on the ACS and that is caused by the ACS config.

perhaps this SOL can help you further: http://support.f5.com/kb/en-us/solutions/public/15000/500/sol15596.html

else you can also just contact support.