Both our production and non-production service desk applications use SSO. User connects to application VIP, which redirects users to the SSO VIP on 443. The F5 configuration for these two environments are identical: SSL bridging with default Client SSL profile as parent. No customizations except for the certificate/key/bundle. However in the non-prod environment, the SSL handshake cannot complete. tcpdump shows a fatal error, certificate unknown, even though this is the same cert/key on the SSO server. When I browse directly to the SSO VIP, the application works as expected. Currently the work-around is to have the non-prod ITSD application server bypass the F5 and go directly to the SSO app server rather than the F5.

Is the SSO VIP doing client certificate authentication? Is there an application behind that you're re-encrypting to? Is the above a client side or server side capture?

Kevin, my understanding is that this setting is on the Client SSL profile? The Client Certificate is set to ignore. We are SSL bridging to the SSO app server. The above is clientside.

Okay, so to be clear, "SSL Bridging" generally defines where the F5 both decrypts and re-encrypts the traffic to the server, which requires both client and server SSL profiles. If I can assume that there is NO client certificate auth going on, on either side, then the "certificate unknown" error is going to come from the client after consuming the server's certificate. Browsers aren't usually that strict about bad server certificates. Worst case you'll get a certificate warning and click-to-continue. So then I'd ask, Does the app server require HTTPS?Do you have a server SSL profile also applied?And if yes to the above, which server SSL profile are you using?

The application requires HTTPS The default serverssl profile is applied to the virtual server.

Okay, so I would then ask that you SSLDUMP on both sides. ssldump -AdNn -i [client or server side VLAN] port 443 This will show you if there's any issues in the respective TLS handshakes. Are there ANY OTHER differences between the environments? Are you using the same browser version in both?

SSL handshake failure during SSO

11 Replies

Kevin_Stewart
Employee
Aug 29, 2018
Is the SSO VIP doing client certificate authentication?

Is there an application behind that you're re-encrypting to?

Is the above a client side or server side capture?
Zuke
Cirrostratus
Aug 29, 2018
Kevin, my understanding is that this setting is on the Client SSL profile? The Client Certificate is set to ignore.

We are SSL bridging to the SSO app server.

The above is clientside.
Kevin_Stewart
Employee
Aug 29, 2018
Okay, so to be clear, "SSL Bridging" generally defines where the F5 both decrypts and re-encrypts the traffic to the server, which requires both client and server SSL profiles. If I can assume that there is NO client certificate auth going on, on either side, then the "certificate unknown" error is going to come from the client after consuming the server's certificate.

Browsers aren't usually that strict about bad server certificates. Worst case you'll get a certificate warning and click-to-continue.

So then I'd ask,

Does the app server require HTTPS?
Do you have a server SSL profile also applied?
And if yes to the above, which server SSL profile are you using?
Zuke
Cirrostratus
Aug 29, 2018
The application requires HTTPS

The default serverssl profile is applied to the virtual server.
Kevin_Stewart
Employee
Aug 29, 2018
Okay, so I would then ask that you SSLDUMP on both sides.

ssldump -AdNn -i [client or server side VLAN] port 443

This will show you if there's any issues in the respective TLS handshakes.

Are there ANY OTHER differences between the environments? Are you using the same browser version in both?

Zuke

Cirrostratus

Aug 31, 2018

Kevin, same browser in both.

Here is the output on the current config:

3 1  0.0011 (0.0011)  C>SV3.3(198)  Handshake
    ClientHello
        Version 3.3 
        random[32]=
        5b 89 77 3d 0b b2 cc 13 45 41 6f 92 af bb 53 51 
        84 7b 22 0c 4b cc 71 cf 17 9d 1d 2b 77 01 cc b8 
        cipher suites
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_RSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA
        TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_RSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        TLS_EMPTY_RENEGOTIATION_INFO_SCSV
        compression methods
                NULL
3 2  0.0012 (0.0000)  S>CV3.3(2)  Alert
    level           fatal
    value           handshake_failure
3    0.0012 (0.0000)  S>C  TCP FIN
3    0.0016 (0.0004)  C>S  TCP FIN

Here is what I see when the dev application owner points the redirect at my lab virtual server, which has the clientssl profile that is identical to the known-good production setup.

1 1  0.0012 (0.0012)  C>SV3.3(161)  Handshake
    ClientHello
        Version 3.3 
        random[32]=
        5b 89 7a b6 35 98 e1 be de 21 a4 60 62 ef 00 9e 
        93 95 37 7c 4f db dc 02 c1 c4 0d ec 53 fe fb 8e 
        cipher suites
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_RSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA
        TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_RSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        TLS_EMPTY_RENEGOTIATION_INFO_SCSV
        compression methods
                NULL
ERROR: Length mismatch

Kevin_Stewart
Employee
Aug 31, 2018
The second one is truncated with the "Length mismatch" error, but can I assume the second one works?

In any case, what you're seeing here is the server side rejecting the client right after the client hello message. The only thing that generally happens in this message is the client sending a preferred TLS protocol, a list of supported ciphers, and any number of TLS extensions (ex. SNI, EC points, etc.).

There is something in the client's client hello message that the server does not like, and is therefore closing the connection. At this stage it can only be a disagreement in one of these things: TLS protocol, ciphers, TLS extensions. I don't actually see the TLS extensions in the capture though.

So to clarify, who is the client and who is the server in this capture?
Zuke
Cirrostratus
Aug 31, 2018
Kevin, both situations result in a failure.

In both instances, the F5 is the server.

I asked the customer if he can change the ciphers on the application servers, since I'm not allowed to make a change to my F5 configuration. Here is the config of the parent clientssl profile showing the defined ciphers, which I believe don't match up with any ciphers in the client hello

ltm profile client-ssl /Common/default_cert_clientssl_profile { app-service none cert /Common/default.crt cert-key-chain { default { cert /Common/default.crt key /Common/default.key } } chain none ciphers ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-SHA256:AES256-G CM-SHA384:AES256-SHA256:!SHA1:!SSLv3:!TLSv1:!TLSv1_1 defaults-from /Common/clientssl inherit-certkeychain true key /Common/default.key passphrase none renegotiation disabled }

When I compare the ciphers allowed in the clientssl profile and what was sent in the Client Hello packet, it looks to me like there are no ciphers that both sides agree upon.

Client Hello:

TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
TLS_RSA_WITH_AES_128_CBC_SHA256
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
TLS_RSA_WITH_AES_128_CBC_SHA
TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_RSA_WITH_AES_128_CBC_SHA
TLS_DHE_DSS_WITH_AES_128_CBC_SHA
TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
TLS_RSA_WITH_AES_128_GCM_SHA256
TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_RSA_WITH_3DES_EDE_CBC_SHA
TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
TLS_EMPTY_RENEGOTIATION_INFO_SCSV

Client SSL profile:

ECDHE-RSA-AES256-GCM-SHA384
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-RSA-AES256-SHA384
ECDHE-ECDSA-AES256-SHA384
DHE-RSA-AES256-GCM-SHA384
DHE-RSA-AES256-SHA256
AES256-GCM-SHA384:AES256-SHA256
!SHA1
!SSLv3
!TLSv1
!TLSv1_1

Zuke

Cirrostratus

Sep 04, 2018

Alright, so we got a new SSL certificate and my theory does not appear to be correct. The clientssl parent profile has been changed to the clientssl-insecure-compatible

Here is a copy of the SSLDUMP:

New TCP connection 22: 192.168.105.92(59853) <-> 192.168.28.140(443)
22 1  0.0012 (0.0012)  C>SV3.3(161)  Handshake
    ClientHello
        Version 3.3 
        random[32]=
        5b 8e d9 0f 64 f5 32 7b 1c 0b fc 92 41 a8 82 7e 
        cb f4 c2 0b 6a 7f f6 f9 30 0a f8 5c f6 85 17 bc 
        cipher suites
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_RSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA256
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA256
        TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        TLS_RSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA
        TLS_ECDH_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        TLS_DHE_DSS_WITH_AES_128_CBC_SHA
        TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_RSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256
        TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
        TLS_DHE_DSS_WITH_AES_128_GCM_SHA256
        TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA
        TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA
        TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA
        TLS_EMPTY_RENEGOTIATION_INFO_SCSV
        compression methods
                NULL
22 2  0.0036 (0.0024)  S>CV3.3(87)  Handshake
    ServerHello
        Version 3.3 
        random[32]=
        75 aa a3 00 16 c9 d3 98 56 56 25 51 f6 1c 62 bc 
        2b a1 99 ea 0b 79 9b d9 9b b8 df c8 86 e8 7c 24 
        session_id[32]=
        aa 00 6c db 7f c5 60 3f 93 c9 50 e3 e6 0d 08 c7 
        34 e2 db 94 60 26 d7 30 42 be 6a a8 b6 9d 58 d7 
        cipherSuite         TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
        compressionMethod                   NULL
22 3  0.0036 (0.0000)  S>CV3.3(4491)  Handshake
    Certificate
        Subject
        CN=itsdtest.example.com
        Issuer
        DC=com
        DC=example
        CN=InternalCA
        Serial         5a 00 00 80 b0 f0 2d 42 8b 48 43 0b bf 00 01 00 
        00 80 b0 
        Extensions
        Extension: X509v3 Key Usage
        Extension: X509v3 Subject Key Identifier
        Extension: X509v3 Subject Alternative Name
        Extension: X509v3 Authority Key Identifier
        Extension: X509v3 CRL Distribution Points
        Extension: Authority Information Access
        Extension: 1.3.6.1.4.1.311.21.7
        Extension: X509v3 Extended Key Usage
        Extension: 1.3.6.1.4.1.311.21.10
        Subject
        DC=com
        DC=example
        CN=InternalCA
        Issuer
        CN=InternalCAroot
        Serial         44 00 00 00 03 5e eb 3e 5c 75 84 80 b2 00 00 00 
        00 00 03 
        Extensions
        Extension: 1.3.6.1.4.1.311.21.1
        Extension: 1.3.6.1.4.1.311.21.2
        Extension: X509v3 Subject Key Identifier
        Extension: 1.3.6.1.4.1.311.20.2
        Extension: X509v3 Key Usage
        Extension: X509v3 Basic Constraints
                    Critical
        Extension: X509v3 Authority Key Identifier
        Extension: X509v3 CRL Distribution Points
        Extension: Authority Information Access
        Subject
        CN=InternalCAroot
        Issuer
        CN=InternalCAroot
        Serial         1c 53 ec 09 e5 3f 49 ab 46 69 30 03 1c 2c eb 61 
        Extensions
        Extension: X509v3 Key Usage
        Extension: X509v3 Basic Constraints
                    Critical
        Extension: X509v3 Subject Key Identifier
        Extension: 1.3.6.1.4.1.311.21.1
22 4  0.0036 (0.0000)  S>CV3.3(333)  Handshake
    ServerKeyExchange
22 5  0.0036 (0.0000)  S>CV3.3(4)  Handshake
    ServerHelloDone
22 6  0.0042 (0.0006)  C>SV3.3(2)  Alert
    level           fatal
    value           certificate_unknown
22    0.0042 (0.0000)  C>S  TCP FIN
22    0.0043 (0.0000)  S>C  TCP FIN

Kevin_Stewart
Employee
Sep 04, 2018
Zuke, you may have answered this but I didn't see it. Can I assume this is a non-browser client? A browser would usually issue a certificate warning to the user and allow the user to bypass the error. A browser client would likely never send this:

22 6 0.0042 (0.0006) C>SV3.3(2) Alert level fatal value certificate_unknown

which basically implies that the client (C>S) does not trust the itsdtest.example.com server certificate.

Forum Discussion

SSL handshake failure during SSO

11 Replies

Recent Discussions

Chrome V 124+ on MacOS - Virtual Server Access Issue

F5 Rseries HA

Need advise to setup a policy on F5

F5 Rseries R2600 Tenant sizing

Can iRule mask the payload content on event logs of security

Related Content

BIG-IP Proxy SSL 12.1 Handshake Failure

Monitoring Failure OAuth request

TCP Internals: 3-way Handshake and Sequence Numbers Explained

SSL Profiles Part 1: Handshakes

SSL handshake failure using serverssl (F5 and Citrix Netscaler)