F5 BIGIP VE with Virtual Forwarding Server dropped some DNS requests...
Hi all,
Last week I had a very strange issue. We have an F5 VE LTM Active/Standby Cluster pair. The setup is running stable for more than 4 months now. In this environment the F5 VE acts as an default gateway towards the internet for some servers. To achieve this I configured an special forwarding virtual server with FastL4 profile for any protocol.
Last Sunday our operation team recognised that there was an issue with DNS requests running through the F5 VE. Note: We do NOT use any DNS (GTM) function on the F5 VE, so DNS traffic just went through the F5. The funny thing was when I did a tcpdump that 3 of 5 DNS requests were answered correctly. For the other 2 requests I never saw a request going out on the outer F5 interface (Internet-Leg). I check several logs and also performance statistics (CPU, Memory, Sessions, etc.) but everything was just normal for non-business hours during a weekend.
After some further testing/debugging I decided to do a Failover to the standby member. Just after the failover happened the DNS issue was gone and all DNS requests where answered properly.
I already opened a case at F5 and uploaded QKVIEW files. F5 support is still analysing the data but was not able to find the root cause so far. F5 support said that one problem could be the Bandwidth-Controller policy (Rate Limit to 50Mbits, burst = 0) I've configured on the forwarding VIP as well. But when the DNS issue came up network usage was very low, only around 10-20 mbits. In the past I did a lot of testing with Bandwidth-Controller and had never such issues before.
How about your experience with FastL4 profiles and any protocols forwarding? Do you have an idea why DNS requests where not forwarded correctly by F5 VE? Is there something special to consider when using Bandwidth-Controller policies on any protocol forwarding virtual servers?
Thanks a lot for your feedback/ideas/suggestions Thrillseeker