Forum Discussion

D__Yutzy_151141's avatar
D__Yutzy_151141
Icon for Nimbostratus rankNimbostratus
Aug 07, 2014

502 Proxy Error

Hello,

 

I've been investigating several 502 Proxy Errors in our environment and can't seem to nail it down. We have a typical architecture of:

 

F5->RP->F5->AS

 

And we are getting the following:

 

HTTP/1.1 502 Proxy Error Date: Wed, 06 Aug 2014 20:18:58 GMT Server: Apache/2.4.9 (Win64) 502 Proxy Error The proxy server received an invalid response from an upstream server.The proxy server could not handle the request GET. Reason: Error reading from remote server

 

It seems quite random although when it happens, it seems tied to the session, then after a period of time gets reset and we don't get it for a while.

 

APM
design
proxy
security

9 Replies

Sort By
  • D__Yutzy_151141's avatar
    D__Yutzy_151141
    Icon for Nimbostratus rankNimbostratus
    Aug 13, 2014
    Wow... 6 days with no response. Scouring the web this seems to be a common problem between Apache and IIS, specifically settings around TimeOut and KeepAlive. Our difference is we have Apache->F5->IIS vs Apache->IIS Again, any help or suggestions is greatly appreciated.
  • David_Larsen_23's avatar
    David_Larsen_23
    Historic F5 Account
    Aug 13, 2014
    Can you provide your F5 configuration information for the VIP to IIS? It could be that modifying the settings within the VIP will help control the timeouts and keepalives.
  • D__Yutzy_151141's avatar
    D__Yutzy_151141
    Icon for Nimbostratus rankNimbostratus
    Aug 14, 2014
    Here is what we have for attributes associated with the VIP •tcp profile - idle timeout 300 seconds, •time wait - 2000 ms •keepalive - 1800 sec
  • Hamish's avatar
    Hamish
    Icon for Cirrocumulus rankCirrocumulus
    Aug 14, 2014

    Do you have some URI's that take longer than 5 minutes (The 300 second idle timeout) to respond (When you get the 502)? I know some developers are keen to ignore standards for HTTP and like to make the client wait for upwards of 30 minutes or so for a response to come back... (Pet peeve of mine).

     

    Failing that... What do the access/error logs say on the Apache that's reporting the error? Is it from your RP or AS?

     

    H

     

  • StephanManthey's avatar
    StephanManthey
    Icon for MVP rankMVP
    Sep 08, 2014

    I´m not sure if I got your latest update right. iRule was removed from the virtual but traffic was still forwarded according to the iRule logic?

    Configuration changes will apply to new connections only. The iRule may still exist in the context of a keepalive connection even it was removed from the virtual server configuration.

    Please make sure to delete all related connections after changing the configuration and before running a new test. (The following command with parameters to specify i.e. a client IP will do the job.) 
    tmsh delete sys conn

    You are using OneConnect? This feature may help you when trying to de-multiplex requests to different targets coming in through a single keepalive connection.

  • D__Yutzy_151141's avatar
    D__Yutzy_151141
    Icon for Nimbostratus rankNimbostratus
    Sep 08, 2014

    I´m not sure if I got your latest update right. iRule was removed from the virtual but traffic was still forwarded according to the iRule logic?

     

    Sorry, I knew it was a complicated workflow to try and describe (with a diagram here. I have, but cannot post):

     

    The custom routing iRule was removed from the app tier F5 and the default, round robin routing was used.

     

  • Rizky_Adrianto_'s avatar
    Rizky_Adrianto_
    Icon for Nimbostratus rankNimbostratus
    Nov 21, 2014

    Yutzy, did this issue resolve? And how you resolve it? I got the same issue and have no idea what makes this proxy error notification appears sporadically..

     

  • D__Yutzy_151141's avatar
    D__Yutzy_151141
    Icon for Nimbostratus rankNimbostratus
    Nov 24, 2014

    The issue was found to be with the iRules we had in place for customized routing in the app tier F5. As I stated above, our basic architecture is:

     

    F5 -> Apache RP -> F5 -> IIS/Glassfish App Servers

     

    These exist in Web and App tiers.

     

    The solution was instead of having a single VIP for Dev, then an iRule to route Dev1, Dev2, Dev3 move to a single VIP for a single URL and remove the routing iRule. We still have not got back to truly figuring out what was causing the problem, but changing to a 1:1 map for URL to VIP worked.

     

    My suspicion is that some sort of state values were getting lost between the external web tier F5, thru the Apache RP and once it hit the app tier F5 and got routed to an app server, the wheels fell off and the result was a 502 Proxy error. This seemingly made it appear as an Apache issue, but I suspect it was some state value between the external F5 and internal F5 getting reset or lost.