What's the protocol?
There's lots of options. Protocols like HTTP or SIP or SOAP have boiler plate monitors in place where you can essentically insert your request and a string to look for in the response.
Proprietary protocols (or protocols not supported directly by BigIP sich as FIX, MQ etc) usually require an external monitor (Although simple ones could be done with a tcp or udp monitor). Basically anything you can write in bash, perl, tmsh etc that the BigIP can run will be called with the monitoring parameters supplied to it. The script can then make the connection, perform the request and check the results, do whaveer additional checks required (Sideband, time of day anything your imagination can come up with), and return the poolmember status.
Custom monitors just need to be executable and placed in the /usr/bin/monitors directory. There's a sample monitor (/usr/bin/monitors/sample_monitor) in there already for you to look at (Shows how to get the IPv4 address from the supplied IPv6 address, etc).
H