BIG-IP Advanced Firewall Manager (AFM) DNS NXDOMAIN Query Attack Type Walkthrough - part two
This is part two of the BIG-IP AFM) DNS NXDOMAIN Query Attack Type Walkthrough article series.
Part one is at https://community.f5.com/t5/technical-articles/big-ip-advanced-firewall-manager-afm-dns-nxdomain-query-attack/ta-p/317656
Reviewing Part One
In part one, we looked at: What and How of DNS NXDOMAIN response and flood definitions
How threat actors generate random dns queries with the use of Domain Generation Algorithms and concepts such as DNS Blackhole, Fast Flux, DNS water torture attack
What information can be used in observing NXDOMAIN response spike
- Reviewing BIG-IP DNS profile statistics collected using a periodic data collection script - provides visibility/statistics on the type of requests and responses the DNS listener processed which are useful when reviewing a recent DNS traffic spike event and understanding the characteristic of the traffic.
- Reviewing sample packet capture during a NXDOMAIN response spike and reducing and zooming in to the data of interest using commands such as capinfos, tshark, sort, uniq, wc
Configuring Detection and Mitigation Thresholds
In this article, we will continue using the information gathered from the NXDOMAIN response flood packet capture and configure BIG-IP Advanced Firewall Manager (AFM) DNS NXDOMAIN Query Attack Type Detection and Mitigation Thresholds.
Configuring BIG-IP AFM DNS NXDOMAIN attack type to mitigate NXDOMAIN response spike
Now that we have information from the sample packet capture and extracted DNS names we can start working on using these information to configure BIG-IP AFM NXDOMAIN attack type Detection and Mitigation thresholds.
The sample packet capture we reviewed ran approx. 13 mins and recorded 60063 packets.
If we divide the number of packets to number of seconds the pcap ran - 60063 p/780s, the number of packets per second is 334 packets per second. Since the packet capture contains only DNS traffic, we can expect it to have both dns requests and responses, which further reduces to 167 packets per second for either dns request or response.
Since NXDOMAIN is a response and the packet capture was taken in a simulated attack to produce NXDOMAIN response, we can use this 167 packets per second as a baseline of what attack traffic looks like.
We should aim for a lower number of packets per second to detect the attack and provide an allowance before we start mitigating the NXDOMAIN response flood.
For the purpose of the demonstration, I have configured a lower detection and mitigation threshold to mitigate the NXDOMAIN response flood. This configuration is on a DNS enabled AFM DoS protection profile that will be applied to a Virtual Server. Dns-dos-protect is the name of the profile in this lab test.
Configuring BIG-IP AFM DNS NXDOMAIN query attack type in a DNS enabled AFM DoS protection profile
- Detection Threshold: 20 EPS
- Mitigation Threshold: 30 EPS
I'll be using 2 test clients to send a flood dns queries to a DNS listener for the hostnames generated thru DGA. As expected, the response for these queries will be NXDOMAIN and the AFM DNS NXDOMAIN Query Attack type will detect the attack as soon as 20 NXDOMAIN responses are observed and will start to drop excess of 30 NXDOMAIN responses.
Here is a sample script to read a file , line by line, that contains DNS names to query. 10.93.56.197 is the DNS listener where the DNS enabled AFM DoS protection profile "dns-dos-protect" is applied to.
root@ubuntu-server1:~# cat nxdig.sh
#!/bin/bash
while read -r line;
do
dig @10.93.56.197 "$line"
done
wait
Here is the DNS listener DoS protection profile configuration. It also shows dos-dns-logging-profile is used as a Log profile.
Here is the dos-dns-logging-profile Log profile profile configuration which only have DNS DoS protection logs enabled and logging to the local-db-publisher (logdb, a mysql db in the BIG-IP)
Using the tmsh show security dos profile <DoS Protection profile name>, we can view the statistics observed by the DoS protection profile per Attack Type. In this test, only NXDOMAIN Query is enabled.
Using the same periodic data capture script when we observed the ltm dns profile statistics, we can capture statistics for the DoS protection profile for review and understanding the phases of the attack being observed.
while true; do date >> /var/tmp/afm_nx_stats; tmsh show security dos profile dns-dos-protect >> /var/tmp/afm_nx_stats; echo "###################" >> /var/tmp/afm_nx_stats; sleep 2; doneRun the dns query flood using the script
while true; do ./nxdig.sh < nx.txt 2>&1; done
Here is a screenshot when the detection threshold of the NXDOMAIN Query attack type was exceeded, see the Attack Status.
Here is when the Attack is being Mitigated
Here is when there is no more Attack being detected
A look at the periodic data capture for the DoS protection profile shows interesting statistics.
Attack Detected - value of 1 means attack is detected which also means the detection threshold was exceeded, value of 0 means no attack currently detected.
Stats - number of packets observed by the Attack type - this is a cumulative value since the BIG-IP booted up
Stats Rate - current number of packets observed by the Attack type - provides an idea of how much of this type of packets currently observed, you can think of this as the current - Events Per Second - EPS of the attack type
Stats 1m - average number of packets per second observed by the Attack type in the last 1m - provides an idea of average number packets of this type every second for the last 1m
The Stats Rate and Stats 1m can provide an idea of how much packets can be seen in the current second and the average per second in the last minute. In non-attack scenario, observing these values shows what normal number of packets may look like. During an attack scenario - detection threshold exceeded - it can provide an idea how much the attack type was seeing. These information can then be used as a basis for setting the mitigation threshold.
For example, it was observed that the Stats 1m value was 21 and during an attack scenario, the Stats Rate value was 245, this is about 12x of the average and the volume appears to be an attack. Depending on the risk appetite of the business, an allowance of 2x of the average number of packets for the attack type is where they want to drop exceeding packets , thus, 42 EPS can be configured for the mitigation threshold.
Do note, setting low detection and mitigation thresholds can cause false positives, triggering detection and mitigation too early. Therefore it is important to understand the traffic characteristic for an attack type.
In the gathered data, we can see here Attack Detected is 1, which means the detection threshold was exceeded. Stats Rate is at 245 which do exceed the 20 EPS detection threshold. Note that there were no Drops stats yet.
Tue Jun 27 07:47:52 PDT 2023
| Attack Detected 1
| ..
| Aggregate Attack Detected 1
| Attack Count 1
....
| Stats 1461
| Stats Rate 245
| Stats 1m 21
| Stats 1h 0
| Drops 0
Tue Jun 27 07:47:56 PDT 2023
Attack Detected 1
...
| Aggregate Attack Detected 1
| Attack Count 1
| Stats 1h Samples 0
| Stats 1516
| Stats Rate 0
| Stats 1m 20
| Stats 1h 0
| Drops 2
Tue Jun 27 07:47:59 PDT 2023
| Attack Detected 1
...
| Aggregate Attack Detected 1
| Attack Count 1
| Stats 1h Samples 0
| Stats 1578
| Stats Rate 0
| Stats 1m 20
| Stats 1h 0
| Drops 4
Drops stats increasing means that an attack is still being mitigated and the volume of the packets is not yet lower that the defined mitigation threshold.Drops - number of packets observed by the Attack type - this is a cumulative value since the BIG-IP booted upDrops Rate - current number of packets dropped by the Attack type
Reviewing DoS stats information in the Reporting DoS DashboardWe have seen the DoS protection profile stats output, now we switch to the GUI and review the same DoS stats information.In the Reporting DoS Dashboard, there are records of the recorded Attacks. The timeframe can be adjusted to find incident of interest. In the testing done, I filtered DNS only and Attack IDs are displayed along with very useful information and statistics.
In this screenshot, Attack ID 2958374472 was selected and relevant statistics are displayed. It was of DNS NXDOMAIN query Attack type/vector it shows how much packets were observed in this attack, which is 235 packets, and dropped packets at 25. Avg PPS - average packets per second for the duration of the attack, similar to the Stats Rate 1m, can also be used as a basis for the Detection and mitigation threshold of the attack type. Domain names observed are also recorded along with the same statistics on the attack.
Configuring BIG-IP AFM DNS NXDOMAIN query attack type in AFM Device DoS Protection
AFM Device Protection also have the DNS NXDOMAIN query attack type. This is a device wide protection and protects the self Ips and Virtual Servers of the BIG-IP. Detection and Mitigation thresholds can be configured the same way, observed the traffic type using the same type of statistics - but this time its Device DoS protection specific.
Here is a sample of tmsh show security dos device-config dos-device-config output and piping it to grep to filter specifically lines for DNS NXDOMAIN Query.
tmsh show security dos device-config dos-device-config | grep -i nxdomain -A 40
Security::DoS Config: DNS NXDOMAIN Query
------------------------------------------------------
Statistics Type Count
Detection Method Static Vector - Inline
Status Ready
Attack Detected 1
...
Aggregate Attack Detected 1
Attack Count 20
Stats 1h Samples 0
Stats 46580
Stats Rate 187
Stats 1m 73
Stats 1h 8
...
Drops 150
...snip..
AutoDetection 137
Mitigation Low 4294967295Similarly, in the GUI, we can observe the states of an attack detected and mitigated by the NXDOMAIN query attack type configured in the AFM Device DoS protection.
Here is the Detection and Mitigation threshold configuration
Attack Detected
We can review the Reporting DoS Dashboard of the Attack events for the Device DoS and review the statistics
Configuring Valid FQDNs in the DNS NXDOMAIN Query Attack type
The DNS NXDOMAIN Query Attack type has a configuration called Valid FQDNs and is described as:
Allows you to create a whitelist of valid fully qualified domain names. In the Add new FQDN field, type a domain name and click Check to see if it is already on the list, click Add to add it to the list, or click Delete to remove it from the whitelist.
Take the name qehspqnmrn[.]fop789[.]loc as an example. Let's assume that this is a valid DNS hostname/FQDN and we do not want DNS NXDOMAIN Query Attack type to drop packets for its response even though it would result in a NXDOMAIN response. We can add it in the Valid FQDNs list from the GUI or tmsh.
Here is the tmsh example, since qehspqnmrn[.]fop789[.]loc is already in the Valid FQDNs list, let's add another FQDN site1[.]fop789[.]loc.
tmsh modify security dos device-config dos-device-config dos-device-vector { dns-nxdomain-query { valid-domains add { site1[.]fop789[.]loc } } }
Here is the tmsh output when listing the DNS NXDOMAIN Query Attack Type including the Valid FQDNs.
tmsh list security dos device-config dos-device-config dos-device-vector { dns-nxdomain-query }
security dos device-config dos-device-config {
dos-device-vector {
dns-nxdomain-query {
...
valid-domains { qehspqnmrn[.]fop789[.]loc site1[.]fop789[.]loc }
}
}
}
To verify that the packets for the FQDN in the Valid FQDNs list are not being dropped, we can look at the Reporting DoS Dashboard.
We can see ongoing attacks are reported and also lists the Domain Names in the attack.
Taking a closer look at the statistics, qehspqnmrn[.]fop789[.]loc in the Domain Name list has NO packet Drops and NO Attack detected. The rest of the DNS names in the list have Drops and Attacks and are being mitigated by the AFM DNS NXDOMAIN query attack type.