NimbostratusIn order to configure the WebScraping, F5 asks to configure the DNS resolver.
Can you explain to me what it is? And can you give me an example of configuration?
Tank's in advance.
Cirrocumulusstboiss, webscraping uses a DNS resolver to verfiy DNS entries of known, good bots, if I recall correctly.
You would need to add a DNS server that the F5 BIG-IP device can reach so it can query DNS requests. See the following solution Configuring a DNS Resolver
Hope this helps,
N
NimbostratusHi,
I have dns at the system level but not in DNS resolver at network level
have you an example of config at network level ?
CirrocumulusDo you get an error when you access the Web Scraping gui to say that you haven't got a DNS server configured? If not then you should be good to go.
NimbostratusI have this error :
This feature will not be fully operational since the DNS Resolver List is empty. To add one, navigate to Network » DNS Resolvers : DNS Resolver List.
Cirrocumulusstboiss, here's my DNS resolver configuration from tmsh:
net dns-resolver dns_resolver {
forward-zones {
. {
nameservers {
8.8.4.4:domain { }
8.8.8.8:domain { }
}
}
}
route-domain 0
}
Hope this helps,
N
EmployeeHi stboiss,
I believe I can provide some extra insight on this. It is correct that WebScraping requires a DNS resolver. The DNS Resolver is basically used to cache DNS responses. The Forward Zone within that resolver is then used to forward DNS requests that aren't already found in the cache. So you should be able to create a resolver with a Forward Zone that has a name that is just "." like you see in nathe's comment with his configuration. Then just add your DNS server IPs to the Resolver as well. This should effectively cause the Forward Zone to forward all DNS requests to your DNS servers so long as there is no response already cached by the Resolver.
For more information on forward zones and how they match the "." you can review the "About forward zones" and "Longest match" sections of the following manual chapter.