Forum Discussion

Mathew_Loesch's avatar
Mathew_Loesch
Icon for Nimbostratus rankNimbostratus
Mar 03, 2015

High Speed Logging event correlation

I have created a request-log profile in v11.4.x. I am looking to create/include a unique marker so I can correlate the request event with the corresponding response event. I currently have the client IP and ephemeral port included in both log values, so I can track the conversation. But I want to solidify which response, belongs to which request. Is there log value that can be included to provide that correlation?

 

5 Replies

  • With the logging profile, I don't know that there is a way to assign a unique ID to each request/response entry, but you could accomplish this with an iRule that handles your HSL. You would just need to set a variable in the HTTP_REQUEST even and log it, then use that variable in the HTTP_RESPONSE event to write your log. You could try setting an ID with somethign liek this,

    set ID [md5 "[clock seconds][IP::client_addr][TCP::client_port]"]

  • Do you have an example of what that iRule may look like? I am not certain if I can call the same profile in HTTP_REQUEST and HTTP_RESPONSE. Or do i have to create two different profiles, and call them in the appropriate event?? Thanks...

     

    • Brad_Parker's avatar
      Brad_Parker
      Icon for Cirrus rankCirrus
      You could try something like this, but I haven't tested it myself yet. when HTTP_REQUEST { set hsl [HSL::open -proto UDP -pool ] set ID [md5 "[clock seconds][IP::client_addr][TCP::client_port]"] HSL::send $hsl "$ID,[clock format [clock seconds] -gmt],[IP::client_addr],[TCP::client_port],[HTTP::request]" } when HTTP_RESPONSE { HSL::send $hsl "$ID,[clock format [clock seconds] -gmt],[IP::client_addr],[TCP::client_port],[HTTP::status]" }
    • Mathew_Loesch's avatar
      Mathew_Loesch
      Icon for Nimbostratus rankNimbostratus
      As part of the output cycle of the profile, it appears that there are groups of request and response entries. Can you tell me if all request and response are tied together? Meaning, when I see a request from a specific client, is it always true that the next response from that client is related to the immediately previous request.
    • Hamish's avatar
      Hamish
      Icon for Cirrocumulus rankCirrocumulus
      That's not guaranteed when using HSL... order of delivery is not preserved. Even if using TCP with HSL, you can get multiple connections and HSL uses the first available (or if it would have to queue, then it'll open new connection). Umm... Is request number available in the logging profile template? Hold on... No they're not... At least in 11.1 (I can't fin the template for 11.6 ATM, but I don't think it's changed anyway). However request_number IS available in an iRule [HTTP::request_num] that's unique for a connection... So if you log the client IP and tcp port and the request num, that should be unique for a few minutes at least (Until the client re-uses the src port which SHOULD be at least 2 minutes). You can tie up a request with a response there. But if you just log everything on the response (In an iRule), you won't even have that problem... If it's info that's only available on the request, just save it in the iRule HTTP_REQUEST event, and reference it in the HTTP_RESPONSE event. Back to a log profile however, you CAN log arbitrary headers... So if there's a header that has a session ID in it you could log that... (Undelineated strings return the value of the respective header. - this is documented in 11.1, so should be available in pretty much all versions) H