Forum Discussion

R_Marc's avatar
R_Marc
Icon for Nimbostratus rankNimbostratus
Jul 14, 2014

Oracle RAC behind LTM

I'm trying to get Oracle RAC integrated with our LTM.

The way JDBC works with RAC, from what I've seen, is the JDBC client connects to one of the RAC SCAN IP's, then there is a redirect sent so they connect to one of the actual DB nodes that currently are serving the service_name request.

In order to make DBA's happy, I need to make LTM behave similarly. The reason we are putting this behind LTM is to facilitate database failover between data centers, which RAC doesn't do very well, at least not in my experience.

So, here's what I've set up which mostly works, but I'm stuck on what last bit.

virtual 1 goes to RAC SCAN IP:Port(s) virtual 2 goes to all the individual nodes.

When you connect to virual 1, I replace the redirect to the individual node with the IP of virtual 2. I add in the following string (REAL=) that I intercepted from the RAC redirect, so I can extract the node that RAC wanted to send the client too. This last bit is where I have the problem.

The client gets the redirect information just fine, but when the JDBC client connects to virutal2, the connect string is truncated exactly by the amount I added to the connect string:

Here's an example of the redirect without that "REAL" info added (this is from a tcpdump):

@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.111.151.151)(PORT=1527)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.111.145.160)(PORT=1527))(CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=x012345))(SERVICE_NAME=INT_SERVICE_ONLINE_SERV)(SERVER=dedicated)(INSTANCE_NAME=mysid3)))

The subsequent connect to virt 2 looks like this:

(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.111.145.160)(PORT=1527))(CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=x012345))(SERVICE_NAME=INT_SERVICE_ONLINE_SERV)(SERVER=dedicated)(INSTANCE_NAME=mysid3)))

Here's what a "BAD" connection looks like (where I'm inserting the "REAL" ip):

@(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=10.111.151.151)(PORT=1527)))
(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.111.145.162)(PORT=1527)(REAL=10.111.145.191))(CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=x012345))(SERVICE_NAME=INT_SERVICE_ONLINE_SERV)(SERVER=dedicated)(INSTANCE_NAME=mysid3)))

But then the subsequent connection is truncated:

(DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.158.145.162)(PORT=1527)(REAL=10.158.145.191))(CONNECT_DATA=(CID=(PROGRAM=JDBC Thin Client)(HOST=__jdbc__)(USER=e018781))(SERVICE_NAME=INT_WALLET_ONLINE_SERV)(SERVER=dedicated)(IN

The truncation is exactly the same size as the about of data I added.

Here is the iRule I'm using (it's a work in progress, don't be mean...A lot was just cut and paste from other iRules, I'll clean it up once I get a working prototype):

modify rule rac-monitoring-rule {
    when CLIENT_ACCEPTED {
  log local0. "- [IP::local_addr]"
  set virtaddr [IP::local_addr]
}
when CLIENT_DATA {
  log local0. "-"
  if { [TCP::payload] contains "(CONNECT_DATA=" } {
        set ip_match [regexp -all -inline -indices [IP::local_addr] [TCP::payload]]
        log local0. "[TCP::payload]"

        foreach instance $ip_match {
           log local0. "instance = $instance"
           set service_start [lindex $instance 0]
           set original_tcp_length [TCP::payload length]
           TCP::payload replace $service_start 14 [LB::server addr]
           log local0. "[TCP::payload]"
           TCP::payload replace 0 2 [binary format S1 [TCP::payload length]]
        }
  }
  log local0. "[TCP::payload]"
  log local0. "[LB::server addr]:[LB::server port]"
  TCP::release
  TCP::collect
}
when SERVER_CONNECTED {
    log local0. "-"
    TCP::collect
}
when SERVER_DATA {
    log local0. "-"
    log local0. "[TCP::payload]"
    if { [TCP::payload] contains "HOST=" and not ([TCP::payload] contains $virtaddr) } {
        regexp -indices "HOST=" [TCP::payload] index
        set start [ expr {[lindex $index [expr {[llength $index] - 1}]] + 1}]
        regexp -indices {\)\(PORT=} [TCP::payload] index
        set end [lindex $index 0]
        set realip [string range [TCP::payload] $start [expr {$end - 1}] ]
        log local0. "start: $start end: $end $virtaddr (realip: $realip)"
        foreach instance "$start" {
           set service_start [lindex $instance 0]
           set original_tcp_length [TCP::payload length]
           TCP::payload replace $service_start [expr {$end - $start}] "10.111.151.151"
           TCP::payload replace 0 2 [binary format S1 [TCP::payload length]]
           log local0. "2: [TCP::payload]  [TCP::payload length]"
        }
         Add in real to redirect
        regexp -all -indices {\(PORT=[0-9]+\)} [TCP::payload] index
        set start [lindex $index 0]
         log local0. "3: replace $service_start [ expr {[string length "(REAL=$realip)"] - 1} ]"
         TCP::payload replace [expr {[lindex $index 1] + 1 }] 0 "(REAL=$realip)"
         TCP::payload replace 0 2 [binary format S1 [TCP::payload length]]
         log local0. "3: [TCP::payload]  [TCP::payload length]"
         log local0. "Connecting to the real IP"
    }
    TCP::release
    TCP::collect
}
when LB_SELECTED {
  log local0. "-"
  TCP::collect
}
}

If I don't add the "REAL" IP, and force it manually to go to the correct node (there's only one node active at the moment, so that's easy) it works fine.
No RepliesBe the first to reply