Forum Discussion

hankycheng_3513's avatar
hankycheng_3513
Icon for Nimbostratus rankNimbostratus
Jan 09, 2013

Weblog export problem

Hi All

 

I have some question about weblog export.

 

I have set Http Virtual Server into F5 ( BIG-IP 11.2.1 Build 807.0 Hotfix HF1) , then how to export web log using W3C format?

 

 

5 Replies

  • is this usable?

     

     

    Formatted Logging For W3c

     

    https://devcentral.f5.com/wiki/irules.FormattedLoggingForW3c.ashx
  • Tks nitass, I will go to reivew.

     

     

    One more question, how to check the object if I enable optimized-caching? And it will be logging if object caching?

     

     

    Tks!
  • how to check the object if I enable optimized-caching?is this what you are asking?

     

     

    sol13255: Displaying and deleting RAM Cache entries from the command line (11.x)

     

    http://support.f5.com/kb/en-us/solutions/public/13000/200/sol13255.html

     

     

    And it will be logging if object caching? HTTP_RESPONSE is not triggered when object is retrieved from cache. HTTP_RESPONSE_RELEASE is triggered. however, some HTTP:: command is not available such as HTTP::status.

     

     

    HTTP_RESPONSE_RELEASE Wiki

     

    https://devcentral.f5.com/wiki/iRules.HTTP-RESPONSE-RELEASE.ashx
  • Tks nitass, Displaying and deleting RAM Cache is useful for me.

     

     

    On HTTP_RESPONSE_RELEASE , it cannot be trigger HTTP action like get or HTTP return code ... etc?
  • On HTTP_RESPONSE_RELEASE , it cannot be trigger HTTP action like get or HTTP return code ... etc?http method (e.g. GET) is saved since HTTP_REQUEST event. HTTP::status command is not valid in HTTP_RESPONSE_RELEASE event.

     

     

    HTTP::status Wiki

     

    https://devcentral.f5.com/wiki/irules.http__status.ashx

     

     

    anyway, to have status code in log, you may use 2 virtual servers and call 2nd virtual server from the 1st one.

     

     

    e.g.

     

     

     frontend virtual server (using virtual command and log)
    
    root@(ve11a)(cfg-sync Changes Pending)(Active)(/Common)(tmos) list ltm virtual bar-frontend
    ltm virtual bar-frontend {
        destination 172.28.20.14:80
        ip-protocol tcp
        mask 255.255.255.255
        profiles {
            http { }
            tcp { }
        }
        rules {
            myrule-frontend
        }
        source 0.0.0.0/0
        source-address-translation {
            type automap
        }
        vlans-disabled
    }
    root@(ve11a)(cfg-sync Changes Pending)(Active)(/Common)(tmos) list ltm rule myrule-frontend
    ltm rule myrule-frontend {
        when HTTP_REQUEST {
       
        Save Request Side Information
       
       set http_request "\"[HTTP::method] [HTTP::uri] HTTP/[HTTP::version]\""
       set http_request_time [clock clicks -milliseconds]
       set http_user_agent "\"[HTTP::header User-Agent]]\""
       set http_host [HTTP::host]
       set http_username [HTTP::username]
       set client_ip [IP::remote_addr]
       set client_port [TCP::remote_port]
       set http_request_uri [HTTP::uri]
       set referer [HTTP::header value referer]
       virtual bar-backend
       log local0. "--"
    }
    when HTTP_RESPONSE {
       set response_time [expr [clock clicks -milliseconds] - $http_request_time]
       set virtual [virtual]
       set content_length 0
       if { [HTTP::header exists "Content-Length"] } {
          set content_length [HTTP::header "Content-Length"]
       }
       set lb_server "[LB::server addr]:[LB::server port]"
       if { [string compare "$lb_server" ""] == 0 } {
          set lb_server ""
       }
       set status_code [HTTP::status]
       set content_type [HTTP::header "Content-type"]
       set log_msg ""
       append log_msg "virtual=$virtual "
       append log_msg "client_ip=$client_ip "
       append log_msg "client_port=$client_port "
       append log_msg "lb_server=$lb_server "
       append log_msg "host=$http_host "
       append log_msg "username=$http_username "
       append log_msg "$http_request_uri "
       append log_msg "request=$http_request "
       append log_msg "server_status=$status_code "
       append log_msg "content_type=$content_type "
       append log_msg "content_length=$content_length "
       append log_msg "resp_time=$response_time "
       append log_msg "user_agent=$http_user_agent "
       append log_msg "referer=$referer"
       log  local0. $log_msg
       log local0. "--"
       log local0. $log_msg
    }
    }
    
     backend virtual server (using optimized-caching profile)
    
    root@(ve11a)(cfg-sync Changes Pending)(Active)(/Common)(tmos) list ltm virtual bar-backend
    ltm virtual bar-backend {
        destination 1.1.1.1:80
        ip-protocol tcp
        mask 255.255.255.255
        pool foo
        profiles {
            http { }
             { }
            tcp { }
        }
        rules {
            myrule-backend
        }
        source 0.0.0.0/0
        source-address-translation {
            type automap
        }
        vlans-disabled
    }
    root@(ve11a)(cfg-sync Changes Pending)(Active)(/Common)(tmos) list ltm rule myrule-backend
    ltm rule myrule-backend {
        when HTTP_REQUEST {
            log local0. "--"
    }
    when HTTP_RESPONSE {
            log local0. "--"
    }
    }
    
     1st request (object is not cached yet)
    
    [root@ve11a:Active:Changes Pending] config  tail -f /var/log/ltm
    Jan 10 13:12:15 ve11a info tmm1[11170]: Rule /Common/myrule-frontend : --
    Jan 10 13:12:15 ve11a info tmm1[11170]: Rule /Common/myrule-backend : --
    Jan 10 13:12:15 ve11a info tmm1[11170]: Rule /Common/myrule-backend : --
    Jan 10 13:12:15 ve11a info tmm1[11170]: Rule /Common/myrule-frontend : --
    Jan 10 13:12:15 ve11a info tmm1[11170]: Rule /Common/myrule-frontend : virtual=/Common/bar-frontend client_ip=172.28.19.251 client_port=34313 lb_server=1.1.1.1:80 host=172.28.20.14 username= request="GET / HTTP/1.1" server_status=200 content_length=89 resp_time=6 user_agent="curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5]" referer=
    
     object is cached
    
    [root@ve11a:Active:Changes Pending] config  tmsh show ltm profile ramcache optimized-caching
    Ltm::Ramcaches /Common/optimized-caching
    
    Host: 172.28.20.14
    URI : /
    --------------------------------------
      Source Slot/TMM  1/0
      Owner Slot/TMM   1/1
      Rank             1
    
      Size (bytes)     353
      Hits             0
      Received         2013-01-10 05:12:15
      Last Sent        2013-01-10 05:12:15
      Expires          2013-01-11 05:43:58
      Vary Type        none
      Vary Count       1
      Vary User Agent  none
      Vary Encoding    none
    Total records returned: 1
    
     2nd request
     you can see HTTP_RESPONSE in myrule-backend is not executed because object is retrieved from cache
    
    [root@ve11a:Active:Changes Pending] config  tail -f /var/log/ltm
    Jan 10 13:12:40 ve11a info tmm[11170]: Rule /Common/myrule-frontend : --
    Jan 10 13:12:40 ve11a info tmm[11170]: Rule /Common/myrule-backend : --
    Jan 10 13:12:40 ve11a info tmm[11170]: Rule /Common/myrule-frontend : --
    Jan 10 13:12:40 ve11a info tmm[11170]: Rule /Common/myrule-frontend : virtual=/Common/bar-frontend client_ip=172.28.19.251 client_port=34314 lb_server=1.1.1.1:80 host=172.28.20.14 username= request="GET / HTTP/1.1" server_status=200 content_length=89 resp_time=0 user_agent="curl/7.15.5 (i686-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5]" referer=