Application Request Parsing

Let's imagine you've been given the task to interrogate all form data that is passed into your web application for the purpose of auditing and validation.  This logic is often built into the application but often security flaws are found and a fix to block that flaw is required before the development team is able to fix the hole in their application. 

This article will illustrate how to write an iRule to interrogate GET or POST form data to allow for auditing and policy enforcement.

HTTP application requests can take the format of HTTP GET and POST commands (well, there are others, but these are the most common).  With a GET command, all parameter names and and values are appended to the URI separated by a question mark.

GET /index.html?p1=pv1&p2=pv2 HTTP/1.1
Host: www.foo.com

With  a POST command, the parameters are not appended to the URI, but sent as part of the HTTP payload with a size specified with the "Content-Length" header.

POST /index.html HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Host: www.foo.com
Content-Length: 13
Expect: 100-continue

p1=pv1&p2=pv2

Applications also use HTTP headers to pass information around.  This can be in the form of HTTP cookies, authentication tokens, mime types, content lengths and other data.  Since headers are often tied to applications, I'l throw in some code to inspect the HTTP headers as well.

To inspect the HTTP headers, you'll want to use the HTTP::header command to extract the header names as well as their associated values as illustrated here.

when HTTP_REQUEST {
  # Inspect Headers
  set names [HTTP::header names]
  foreach name $names {
    set val [HTTP::header value $name]
    log local0. "    $name: $val"
  }
}

Next, you'll likely want to do some URI parsing to determine which application the request is going to.  This code uses the URI::path depth subcommand to determine the number of directories (the depth) in the URI (retrieved with the HTTP::uri command).  It then enters a for loop and extracts each directory, again with the URI::path command.  Finally it uses the URI::basename command to extract the trailing file component from the URI.

when HTTP_REQUEST {
  # Inspect URI
  set depth [URI::path [HTTP::uri] depth]
  for {set i 1} {$i <= $depth} {incr i} {
    set dir [URI::path [HTTP::uri] $i $i]
    log local0. "    dir\[$i\]: $dir"
  }
  log local0. "    Basename: [URI::basename [HTTP::uri]]"
}

Once the URI has been parsed, you'll want to determine whether the request is a GET or POST request.  This can be done with the HTTP::method command

when HTTP_REQUEST {
  switch [HTTP::method] {
    "GET" {
      log local0. "GET Request"
    }
    "POST" {
      log local0. "POST Request"
    }
  }
}

For a GET request, you'll want to use the HTTP::query command along with the TCL split command to extract the query string arguments from the end of the URI

when HTTP_REQUEST {
  # Inspect Query String
  set namevals [split [HTTP::query] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
    log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}

For a POST request, you'll have to do a little more work to ensure that the body of the request containing the POST data has been received.  For small payloads, the full body has likely been retrieved from within the HTTP_REQUEST event but it's better to be safe than sorry.  You'll want to issue a HTTP::collect with the value supplied in the "Content-Length" header and process the payload in the HTTP_REQUEST_DATA event.

One thing to mention here, is that it might be good form to verify the Content-Type to verify it's value is "application/x-www-form-urlencoded"  Other applications make use of HTTP POST commands (XML-RPC, SOAP, etc) so to make sure it's a HTML.

when HTTP_REQUEST {
  if { [HTTP::header Content-Type] eq "application/x-www-form-urlencoded" } {
    HTTP::collect [HTTP::header Content-Length]
  }
}

when HTTP_REQUEST_DATA {
  set namevals [split [HTTP::payload] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
   log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}



You now have all the tools to extract and inspect the HTTP Headers, HTTP query string (for GET Requests) and HTTP post data (for POST requests).  If you wanted to put it all together in one iRule, it might look something like this:

when HTTP_REQUEST {

  # Inspect Headers
  log local0. "============================"
  log local0. "<<< HTTP Headers >>>"
  set names [HTTP::header names]
  foreach name $names {
    set val [HTTP::header value $name]
    log local0. "    $name: $val"
  }

  # Inspect URI
  log local0. "<<< URI >>>"
  log local0. "    HTTP::uri: [HTTP::uri]"
  log local0. "    HTTP::path: [HTTP::path]"
  set depth [URI::path [HTTP::uri] depth]
  for {set i 1} {$i <= $depth} {incr i} {
    set dir [URI::path [HTTP::uri] $i $i]
    log local0. "    dir\[$i\]: $dir"
  }
  log local0. "    Basename: [URI::basename [HTTP::uri]]"

  switch [HTTP::method] {
    "GET" {
      # Inspect Query String
      log local0. "<<< Query Information >>>"
      log local0. "    HTTP::query: [HTTP::query]"
      set namevals [split [HTTP::query] "&"]
      for {set i 0} {$i < [llength $namevals]} {incr i} {
        set params [split [lindex $namevals $i] "="]
        log local0. "    [lindex $params 0] : [lindex $params 1]"
      }
    }
    "POST" {
      if { [HTTP::header Content-Type] eq "application/x-www-form-urlencoded" } {
        log local0. "<<< Post Data Information >>>"
        HTTP::collect [HTTP::header Content-Length]
      }
    }
  }
}

when HTTP_REQUEST_DATA {
  log local0. "    POST Data: [HTTP::payload]"
  set namevals [split [HTTP::payload] "&"]
  for {set i 0} {$i < [llength $namevals]} {incr i} {
    set params [split [lindex $namevals $i] "="]
    log local0. "    [lindex $params 0] : [lindex $params 1]"
  }
}

Well, now you have all the tools you need to interrogate HTTP form requests.  Inspect away...

Published Jul 26, 2007
Version 1.0

Was this article helpful?

6 Comments

  • I'm using this as a learniung tool, and unfortunately, it appears I'm not learning all that quickly.

     

     

    when I run the 'Inspect Headers portion of this rule, I get this returned in my ltm logs:

     

     

    Aug 31 15:46:31 tmm tmm[751]: 01220001:3: TCL error: Rule Inspect_Headers

     

    - Illegal argument. Illegal argument. Illegal argument. Illegal argument. Ille (line 5)

     

    invoked from within "HTTP::header values $name"

     

    ("foreach" body line 2)

     

    invoked from within "foreach name $names

     

    { set val [HTTP::header values $name] log local0. " $name: $val" }"

     

     

     

  • I'm still on break over the holiday. My dev network at works seems to be not-accessible from my VPN connection so I'll get to verifying your issue as soon as I get back into the office tomorrow. What version of BIG-IP are you seeing this error on? I verified it on 9.4 before posting this tech tip.

     

     

    -Joe
  • I've just verified that this code works on the latest revision of BIG-IP (v9.4.1). Can you add the following line before the "set val" line

     

     

    log local0. "Extracting value for header: '$name'"

     

     

    And post the log results as well as your version of the BIG-IP. The HTTP::header command has been around since day 1 of v9.0 and we haven't seen this problem before. Could it possibly be a typo?

     

     

    Actually, if you could post this as a new question on the v9 iRule forum, that will make it more accessible to everyone.

     

     

    -Joe
  • Thanks, Joe -

     

    I'll repost to the forums, but here is the info requested.

     

     

    BigIP Version is older... 9.1.0 Build 6.2

     

    I added the requested line to the rule, and the result was:

     

    Sep 5 10:39:15 tmm tmm[751]: Rule Inspect_Headers : ============================

     

    Sep 5 10:39:15 tmm tmm[751]: Rule Inspect_Headers : <<< HTTP Headers >>>

     

    Sep 5 10:39:15 tmm tmm[751]: Rule Inspect_Headers : Extracting value for header: 'Accept'

     

    Sep 5 10:39:15 tmm tmm[751]: 01220001:3: TCL error: Rule Inspect_Headers - Illegal argument. Illegal argument. Illegal argument. Illegal argument. Ille (line 1) invoked from within "HTTP::header values $name" ("foreach" body line 3) invoked from within "foreach name $names { log local0. "Extracting value for header: '$name'" set val [HTTP::header values $name] log local0. "$name: $val" }"

     

  • The issue is due to the fact that the "HTTP::header values" subcommand wasn't available until BIG-IP v9.4. I've updated the tech tip to use the older "HTTP::header value" command.

     

     

    -Joe
  • he issue is due to the fact that the "HTTP::header values" subcommand wasn't available until BIG-IP v9.4. I've updated the te