Cirrus
CirrusHi - thanks for you replying. Sorry CPS refers to connections per second as seen on the stats on LTM - although we are only talking about DNS on the LTM (UDP hence connectionless) - I suppose a better description would be flows per second, but LTM shows them as connections. Unfortunately still have not determined the root cause which started it all off.
Yes it's a full production license - 1Gbps throughput (however 40K cps of DNS relates to only about ~60Mbps of total throughput) . There was a very minor spike in RAM usage but nothing i would have been concerned about. VE has 2 vCPU - busiest cpu was peaking ~50%. I saw similar performance on v11.2 when load testing (we've only just upgraded to 11.4 due to particular features) but at that time put it down to LTM VE limitation due to CPU reaching nearly 100%, it now seems the LTM-VE can use multiple vCPU so an increase in performance like you say.
Licensed Service Rates DNS GTM Effective Rate Limit (RPS) 0 0 Configured Object Count 0 0 Rate Rejected Requests 0 0
all interfaces on VE with LTM do not show any packet drops. Unfortunately we didn't get to check the network stats on ESXi host (ESXTOP) before we got service restored. I am going to try and replicate the conditions in the lab using load testing and see if it's ESXi which is dropping traffic - I have a niggle that it could be related to UDP buffers on ESXi, however much of the information recommends changing these on the VM hosts - but in this case it would mean changing on the LTM, something I'm quite reluctant to do as i assume LTM is all pre-tuned. I'd be interested if anyone has any experience with this.
http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&docTypeID=DT_KB_1_1&externalId=1010071
I've attached a couple of screenshots our the LTM usage stats.