I haven't tested this, but there is a comment in the /config/profile_base.conf for the authentication rules which should help:
When multiple auth http profiles (ldap, radius, tacacs) are simultaneously
configured on a single virtual server, AND-based logic is used by default,
i.e., all authentication methods must succeed for the request to be allowed.
It is also possible to configure OR-based logic, e.g., if either ldap or
radius are successful, allow the request. PAM service configurations could
be manually edited to accomplish this, but a simple iRule can also be used:
Add a custom CLIENT_ACCEPTED rule to the same virtual server and have the
rule set the variable tmm_auth_http_sufficient_successes to 1. Generically,
this variable may be set to the minimum number of successful auth results
that are necessary to permit the request. For example, setting the value
to 2 while ldap, radius, and tacacs profiles are each configured on a
virtual will cause requests to be permitted when at least 2 of these 3
auth methods are successful.
As the client certificate authentication is done via profile, I think you'd need to just set tmm_auth_http_sufficient_successes to 0 for a particular TCP connection if the client cert was valid and set it to 1 if the cert validation failed.
Aaron