Forum Discussion

fluzocapacitor's avatar
fluzocapacitor
Icon for Altocumulus rankAltocumulus
Jun 17, 2024

Querying SNMP data for half-open connections on a virtual server

Hello,

I’m trying to monitor the embryonic/half-open connections for a particular VS on my F5 load balancer using SNMP. 

I’ve read several articles, including ‘K00560557: Using SNMP information related to SynCookie Protection for monitoring potential Syn Flood attack or Unexpected Traffic spikes’ and ‘K25162232: How to get the Virtual Server state using SNMP polling.’ but I did not find how to do it.

Could you please guide me on how to retrieve this information?

Regards,

4 Replies

  • Hi Fluzocapacitor,

     

    Please see the specific OID for your half-open connections 

    1.3.6.1.4.1.3375.2.2.10.2.3.1.13 ltmVirtualServStatEphemeralPktsIn 0 0 The number of packets received ephemeral by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.14 ltmVirtualServStatEphemeralBytesIn 0 0 The number of bytes received ephemeral by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.15 ltmVirtualServStatEphemeralPktsOut 0 0 The number of packets transmitted ephemeral from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.16 ltmVirtualServStatEphemeralBytesOut 0 0 The number of bytes transmitted ephemeral from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.17 ltmVirtualServStatEphemeralMaxConns 0 0 The maximum ephemeral connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.18 ltmVirtualServStatEphemeralTotConns 0 0 The total ephemeral connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.19 ltmVirtualServStatEphemeralCurConns 0 0 The current ephemeral connections to the specified virtual server.

     

    Check a reference for VIP OID  1.3.6.1.4.1.3375.2.2.10.2.3.1.1

    http://oidref.com/1.3.6.1.4.1.3375.2.2.10.2.3.1.1

    OID Name Sub children Sub Nodes Total Description
    1.3.6.1.4.1.3375.2.2.10.2.3.1.2 ltmVirtualServStatCsMinConnDur 0 0 The minimum connection duration in milliseconds among all connections
    through the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.3 ltmVirtualServStatCsMaxConnDur 0 0 The maximum connection duration in milliseconds among all connections
    through the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.4 ltmVirtualServStatCsMeanConnDur 0 0 The average connection duration in milliseconds for all connections
    through the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.5 ltmVirtualServStatNoNodesErrors 0 0 The number of no-node errors. If active server node is zero,
    this counter is incremented.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.6 ltmVirtualServStatClientPktsIn 0 0 The number of packets received by the specified virtual server from client-side.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.7 ltmVirtualServStatClientBytesIn 0 0 The number of bytes received by the specified virtual server from client-side.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.8 ltmVirtualServStatClientPktsOut 0 0 The number of packets sent to client-side from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.9 ltmVirtualServStatClientBytesOut 0 0 The number of bytes sent to client-side from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.10 ltmVirtualServStatClientMaxConns 0 0 The maximum connections from client-side to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.11 ltmVirtualServStatClientTotConns 0 0 The total connections from client-side to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.12 ltmVirtualServStatClientCurConns 0 0 The current connections from client-side to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.13 ltmVirtualServStatEphemeralPktsIn 0 0 The number of packets received ephemeral by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.14 ltmVirtualServStatEphemeralBytesIn 0 0 The number of bytes received ephemeral by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.15 ltmVirtualServStatEphemeralPktsOut 0 0 The number of packets transmitted ephemeral from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.16 ltmVirtualServStatEphemeralBytesOut 0 0 The number of bytes transmitted ephemeral from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.17 ltmVirtualServStatEphemeralMaxConns 0 0 The maximum ephemeral connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.18 ltmVirtualServStatEphemeralTotConns 0 0 The total ephemeral connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.19 ltmVirtualServStatEphemeralCurConns 0 0 The current ephemeral connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.20 ltmVirtualServStatPvaPktsIn 0 0 The total number of hardware accelerated packets received by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.21 ltmVirtualServStatPvaBytesIn 0 0 The total number of hardware accelerated bytes received by the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.22 ltmVirtualServStatPvaPktsOut 0 0 The total number of hardware accelerated packets sent out from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.23 ltmVirtualServStatPvaBytesOut 0 0 The total number of hardware accelerated bytes sent out from the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.24 ltmVirtualServStatPvaMaxConns 0 0 The maximum hardware accelerated connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.25 ltmVirtualServStatPvaTotConns 0 0 The total hardware accelerated connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.26 ltmVirtualServStatPvaCurConns 0 0 The current hardware accelerated connections to the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.27 ltmVirtualServStatTotRequests 0 0 The total number of requests going through the specified virtual server.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.28 ltmVirtualServStatTotPvaAssistConn 0 0 The total number of connections to the specified virtual server, which
    are partially hardware accelerated.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.29 ltmVirtualServStatCurrPvaAssistConn 0 0 The current number of connections to the specified virtual server, which
    are partially hardware accelerated.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.30 ltmVirtualServStatCycleCount 0 0 CPU cycle count since last system restart.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.31 ltmVirtualServStatVsUsageRatio5s 0 0 The percentage of time Virtual Server was busy over the last 5 seconds.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.32 ltmVirtualServStatVsUsageRatio1m 0 0 The percentage of time Virtual Server was busy over the last 1 minute.
    1.3.6.1.4.1.3375.2.2.10.2.3.1.33 ltmVirtualServStatVsUsageRatio5m 0 0 The percentage of time Virtual Server was busy over the last 5 minutes.

     

     

    Please mark it as SOLUTION if your query is answered as it helps others to find a solution as well for similar query.

    Please let me know for any further discussion , i would be glad to assist you on F5 related topics.

    HTH

    🙏

     

  • Hi Fluzocapacitor,

     

    Keeping server side connection open even after the client side connection already been closed is a KNOWN BUG which was fixed in 11.6.x

     

    Known Issue

    Archived - K15973: Server-side TCP connections may remain half-open if the client-side connection closes early

    https://my.f5.com/manage/s/article/K15973

    I dont think there are SNMP MIBs for monitoring half open connections, but you can check in Pools there is a TCP Half Open monitor that can be applied. 

    The f5networks.f5_modules.bigip_monitor_tcp_half_open module can be used to manage F5 BIG-IP LTM TCP half-open monitors. This module is part of the f5networks.f5_modules collection version 1.27.1 and requires BIG-IP software version 12 or higher. Some parameters of the module include:

    • description: A string describing the monitor
    • interval: An integer that specifies how often the monitor instance will run
    • ip: An IP address that is part of the IP/port definition
    • name: The monitor name

    https://docs.ansible.com/ansible/latest/collections/f5networks/f5_modules/bigip_monitor_tcp_half_open_module.html

    You can use the following commands to view and delete the current or active connections

    K53851362: Displaying and deleting BIG-IP connection table entries from the command line

    https://my.f5.com/manage/s/article/K53851362

    K40033505: Explaining the output of tmsh show sys connection

    https://my.f5.com/manage/s/article/K40033505

     

    Examples

     

    tmsh show /sys connection

     

    TMSH command to list all the options & properties with the show /sys connection command

     

    tmsh show /sys connection ?

     

    Options:

    all-properties             Display all properties for the specified items

    default                      Units are determined based on current values

    exa                            All values are displayed in exa-units

    gig                            All values are displayed in giga-units

    kil                             All values are displayed in kilo-units

    meg                           All values are displayed in mega-units

    peta                           All values are displayed in peta-units

    raw                            No conversions are made to any values

    save-to-file                 Output from the command is saved to the specified file. -

                                      This file is placed in /shared. This allows to write a file larger than 2GB.

    tera                            All values are displayed in tera-units

    yotta                          All values are displayed in yotta-units

    zetta                          All values are displayed in zetta-units

    |                                Route command output to a filter

     

    Properties:

    "{"                           Optional delimiter

    age                           Specifies the age, in seconds, of a connection

    cs-client-addr            Specifies the clientside remote address of the active connections

    cs-client-port            Specifies the clientside remote port of the active connections

    cs-server-addr          Specifies the clientside local address of the active connections

    cs-server-port           Specifies the clientside local port of the active connections

    protocol                   Specifies the protocol used for specified connections (for example: tcp, udp)

    ss-client-addr           Specifies the serverside local address of the active connections

    ss-client-port            Specifies the serverside local port of the active connections

    ss-server-addr          Specifies the serverside remote address of the active connections

    ss-server-port          Specifies the serverside remote port of the active connections

     

     

    Using my example output line above we will label each part of the output:

     

    Client Side

    172.10.50.20:30322                       10.10.10.1:443   

    cs-client-addr:cs-client-port             cs-server-addr:cs-server-port

     

    Server Side

    192.168.1.1:30322                         192.40.200.20:30322   

    ss-client-addr:ss-client-port             ss-server-addr:ss-server-port

     

    The protocol used, the age of the connection, and which traffic management microkernel was used

    tcp               14        (tmm: 6)  none

    protocol       age      tmm used

     

     

    How to delete a connection from the F5 BIG-IP connection table?

    To delete active connections in the BIG-IP connection table you can key on any of the property value(s). For example, if I wanted to delete all connections a user was initiating to the BIG-IP you would specify the users IP as the client side client address like this:

     

     

    TMSH command to delete all connections initiated from a specific client IP

     

    tmsh delete /sys connection cs-client-addr  172.10.50.20

     

     

     

    TMSH command to delete ALL connections in a F5 BIG-IP  - Careful, you're going to kill EVERY session.

    tmsh delete /sys connection

     

    Make sure you check out the “all-properties” option, it has a load of great information, including how many bits the connection has pushed, and its’ age & idle-timeout. Lets take a look at the output:

     

     

    TMSH command to show all details about a particular conneciton, including how much traffic, bits in and out, a connection has used 

     

    tmsh show /sys connection cs-client-addr 172-10.50.20 all-properties

     

    172.10.50.20:30322 - 10.10.10.1:443 - 172.10.50.20:30322 - 192.40.200.20:443

    ----------------------------------------------------------------------------

      TMM            6

      Type              self

      Acceleration   none

      Protocol         tcp

      Idle Time       9

      Idle Timeout  300

      Unit ID          0

      Lasthop         /Common/internal 00:0d:3a:3a:25:20

      Virtual Path   10.10.10.1:443

      Conn Id 0

                                   ClientSide           ServerSide

      Client Addr  172.10.50.20:30322    172.10.50.20:30322

      Server Addr      10.10.10.1:443     192.40.200.20:443

      Bits In                    3.0K                1.3K

      Bits Out                  1.3K               3.0K

      Packets In                   4                  3

      Packets Out                 3                  4

     

    HTH
    ✌️

     

    • fluzocapacitor's avatar
      fluzocapacitor
      Icon for Altocumulus rankAltocumulus

      Hi,

      Thank you for the detailed information and the helpful links.

      It’s good to know about the known bug and its fix in version 11.6.x. However, I am not affected by bug K15973 as I am running version 15.1.10.

      I appreciate the suggestion to use the TCP Half Open monitor. I understand that this monitor would help in monitoring the connections between the virtual server and the pool nodes. However, my main interest is in monitoring the connections between the clients and the virtual server. A SYN flood attack would cause issues on the load balancer itself, not on the backend nodes.

      I also appreciate the commands to view and delete active connections.  I have used the following command to check the number of connections:

      show /sys connection cs-server-addr 10.1.50.200%1
      
      Sys::Connections
      10.1.34.120%1:51080   10.1.50.200%1:80  10.1.34.120%1:51080   10.2.75.150%1:80  tcp  16   (tmm: 0)  server-side(none)       none
      10.1.58.99%1:61766    10.1.50.200%1:80  10.1.58.99%1:61766    10.2.92.180%1:80  tcp  28   (tmm: 0)  none                            none
      10.2.85.43%1:51172    10.1.50.200%1:80  10.2.85.43%1:51172    10.2.92.180%1:80  tcp  39   (tmm: 0)  both-sides(none;none)  none
      ...
      10.1.123.45%1:59943   10.1.50.200%1:80  10.1.123.45%1:59943   10.2.75.150%1:80  tcp  2    (tmm: 1)  none                             none
      10.1.87.101%1:55831   10.1.50.200%1:80  10.1.87.101%1:55831   10.2.92.180%1:80  tcp  6    (tmm: 1)  none                               none
      Total records returned: 1000

      Although there are 1000 established connections, my main interest is in half-open connections, as these are the ones that trigger the SYN cookie mechanism. According to the documentation, these half-open connections appear as "Current SYN Cache" in the output of the show ltm virtual <virtual> command.

      Just to clarify, is there a specific OID or SNMP method to directly monitor the number of half-open connections between the clients and the virtual server, or would the TCP Half Open monitor still be the best approach to track this metric?

      Thank you again for your assistance!

      Best regards,