Adding on to Michaels answer above, I modified his iRule to work with O365 open relay. It took me a while to figure out, so for me the trick was to have both a ClientSSL and ServerSSL profile, and use a SMTP profile, not a SMTPS profile.
I'm not sure if this works with client initiated STARTTLS but I suspect it's just combining this with some of the work from https://devcentral.f5.com/questions/smtp-starttls-irule
when CLIENT_ACCEPTED {
No SSL client side, also check no SSL running already on server side
Debug mode
set DEBUG 0
if { $DEBUG } { log local0. "CLIENT_ACCEPTED" }
SSL::disable serverside
SSL::disable
}
when SERVER_CONNECTED {
if { $DEBUG } { log local0. "SERVER_CONNECTED" }
TCP::collect
}
when SERVER_DATA {
Read in responses from remote server into a variable and log to /var/log/ltm
if { $DEBUG } { log local0. "server payload: [string tolower [TCP::payload]]" }
set payload [string tolower [TCP::payload]]
if {$payload starts_with "220" and $payload contains "esmtp"}
{
Listen for remote servers opening 220 and esmtp message
NOTE the ‘if’ statement above may need to be tweaked to except what message the other
side is actually sending in reply. Logs should show this.
Respond with a EHLO to server, most servers require a name after the EHLO as well.
TCP::respond "EHLO F5.mydomain.com\r\n"
TCP::payload replace 0 [TCP::payload length] ""
TCP::release
if { $DEBUG } { log local0. "responded to server with EHLO" }
serverside {TCP::collect}
}
elseif {$payload contains "250-starttls" }
{
Check server responds with "250-starttls", if so, respond with a STARTTLS
TCP::respond "STARTTLS\r\n"
TCP::payload replace 0 [TCP::payload length] ""
TCP::release
if { $DEBUG } { log local0. "Sent the server a STARTTLS" }
serverside {TCP::collect}
}
elseif {$payload contains "220 ready for tls" or $payload contains "220 2.0.0 continue" or $payload contains "220 2.0.0 smtp server ready" }
{
if server gives a 220 response, then start server side ssl profile
NOTE the ‘if’ statement above may need to be tweaked to except what message the other
side is actually sending in reply. Logs should show this.
O365 Edit - O365 returns 220 2.0.0 smtp server ready after enabling TLS
if { $DEBUG } { log local0. "server said he is ready for TLS, enable the SSL profile" }
TCP::payload replace 0 [TCP::payload length] ""
TCP::release
serverside {SSL::enable}
TLS hanshake should now start, which is best seen in wireshark packet captures.
}
}
when SERVERSSL_HANDSHAKE {
This will only trigger if that is completed successfully.
ServerSSL profile will need a certificate to match the outbound IP and DNS name,
and you may want to set the "Server certificate" setting to "require",
and the "Trusted Certificate Authorities" set to "ca-bundle".
if { $DEBUG } { log local0. "SSL handshake completed." }
clientside { TCP::respond "220 SMTP ESMTP Relay F5\r\n" }
SSL::collect
}
when SERVERSSL_DATA {
Log the SMTP responses to see any errors.
if { $DEBUG } { log local0. "server SSL payload: [SSL::payload]" }
SSL::release
SSL::collect
}