Forum Discussion

Maria_Ramirez_5's avatar
Maria_Ramirez_5
Icon for Nimbostratus rankNimbostratus
Jan 05, 2007

memory usage grows up daily in unexpected way

Good morning,

 

 

We have a pair of two LTMs 1500 appliances. They are loadbalancing BEA WEBLOGIC servers and iPlanet servers. We are having problems due to memory. The memory grows up daily (40 Megas per day) in each system. When the memory reaches 260 Megas approximately, the active system reboots. I have a case opened with support and they tell me that in the qkviews, it seems as if the problem is related to the iRules we have configured in order to maintain universal persistence with BEA WEBLOGIC Severs and iPlanet servers.

 

The iRules we have configured are:

 

 

rule Persist_iPlanet {

 

when HTTP_REQUEST {

 

if { [HTTP::cookie exists WEBSRV] } {

 

persist uie [HTTP::cookie WEBSRV];

 

 

}

 

}

 

 

when HTTP_RESPONSE {

 

if { not [HTTP::cookie exists WEBSRV ] } {

 

set ip_srv [IP::server_addr];

 

set valor [ string range $ip_srv [ expr [string last . $ip_srv ] + 1] 15];

 

set val_cookie "WEBSRV=$valor; path=/"

 

HTTP::header insert Set-Cookie $val_cookie;

 

persist add uie $valor;

 

}

 

}

 

}

 

rule Persist_Weblogic {

 

when HTTP_REQUEST {

 

if { [HTTP::cookie exists WLSRV] } {

 

persist uie [HTTP::cookie WLSRV];

 

 

}

 

}

 

 

when HTTP_RESPONSE {

 

if { not [HTTP::cookie exists WLSRV ] } {

 

set ip_srv [IP::server_addr];

 

set valor [ string range $ip_srv [ expr [string last . $ip_srv ] + 1] 15];

 

set val_cookie "WLSRV=$valor; path=/"

 

HTTP::header insert Set-Cookie $val_cookie;

 

persist add uie $valor;

 

}

 

}

 

}

 

rule Adapta_SecureConnection {

 

when HTTP_REQUEST {

 

if { [HTTP::header SecureConnection] equals "true" } {

 

HTTP::header replace "WL-Proxy-SSL" true;

 

}

 

HTTP::header remove SecureConnection;

 

}

 

}

 

Perhaps the problem is because the Persistence table grows and it is not erased properly...?

 

Perhaps the problems is due to the timeouts...if so, when can we configure it and how (in an appropiate way)?

 

Perhaps we have to change some parameters/commands in the iRules?

 

 

We are having this problem from the beginning of the installation and we do not know what to do.

 

I would appreciate your help very much!!

 

Thanks in advance,

 

María

6 Replies

  • Hi Maria,

     

     

    What version are you running? There is a known issue in 9.1.x and 9.2.x where TMM memory for TCL isn't always released. This has been noted in CR70058:

     

     

    If persistence is specified in an iRule (UIE persistence, for instance) and something goes wrong in the connection attempt (like all pool members are down), TMM memory usage will continue to grow.

     

     

    Dev noted this in the following CR's:

     

     

    CR70058 - 9.4.x

     

    CR70119- 9.1.x

     

    CR70120 - 9.2.x

     

     

    You can contact support to request confirmation that this is the issue you are seeing and a hotfix.

     

     

    Aaron
  • Thank you very much Aaron.

     

     

    We are running version 9.2.3.

     

    I will tell your response to support in order to check if they have consider it. I will check the CRs you refer.

     

    I hope it will be solved with an appropiate hotfix.

     

    So...you do not think that the iRules are configured in a bad way...?

     

     

    Thanks
  • No problem... what's your case number? I tried looking but didn't find it.

     

     

    I don't see any glaring issues after a cursory look over the rules.

     

     

    You can measure the memory usage for the bug in CR70058 by running:

     

     

    tmctl umem | grep -i alloc

     

     

    Check the values for umem_alloc_8, umem_alloc_16, and umem_alloc_2504 to see if they grow. To force the failure, you could disable the nodes in the pool--but to see if this is the specific bug you're encountering, you could just append the output to a file and see if the counters increment over the course of a day or two. If they increase as the rule is hit and connections fail, then you've encountered this bug. If not, then I'd suggest you continue working with support to see what is causing the memory leak.

     

     

    Aaron
  • Hello,

     

     

    My support case number is C311800

     

    I will tell my client to do the workaround you tell me in order to check if it is the bug you refer.

     

     

    If you have any other idea...Please tell me.

     

    Thanks
  • Jon

    Did you ever find an answer to your memory leak? I too am seeing a similar memory leak in my iRules in 10.1.0. The memory consumption rate is steady- not tied to traffic load. I'm showing similar numbers in my tmctl output:

     

     

     

    name allocated max_allocated held size

     

    tcl (variable) 376576032 376680192 376576032 1

     

    tcl_ip_addr 19320 30880 770 40

     

    tcl_strcache 235902880 235910784 7372212 32

     

    umem (variable) 16304800 16304800 12375214 1

     

    xfrag 100382720 239679488 116849 2048

     

     

     

     

     

     

  • Hi mrintzler,

     

     

    The issue found in Jon's last issue was with mirroring being enabled but not configured.

     

     

    Can you reply to the questions I had in your previous thread (http://devcentral.f5.com/Community/GroupDetails/tabid/1082223/asg/50/afv/topic/aft/1174720/aff/5/showtab/groupforums/Default.aspx1199102)? Also, I suggest opening a case with F5 Support on suspected memory leaks, as the troubleshooting generally requires access to the full configuration, logs and tech.out's.

     

     

    Aaron