http_process_state_prepend - Invalid Action:0x109010

Here's the code:

when HTTP_REQUEST {
    set DEBUG 2
    set uri [HTTP::uri]

    if {$DEBUG>1} { log local0. "" }
    if {$DEBUG>1} { log local0. "----------------------------------------------" }
    if {$DEBUG  } { log local0. "Path: '[HTTP::path]'" }
    if {$DEBUG>2} { log local0. "Request: [HTTP::request]" }

    switch -glob [HTTP::header value "User-Agent"] {
        "*mirall/*" -
        "*csyncoC/*" -
        "*neon/*" {            
             Disable APM if not already authenticated
            if { not ( [HTTP::cookie exists MRHSession] && [ACCESS::policy result -sid [HTTP::cookie value MRHSession]] equals "allow" ) } {
                ACCESS::disable
            }
            return
        }
        default {
             Check the PATH for authentication exclusions
            switch -glob [string tolower [HTTP::path]] {
                "/remote.php/webdav/*" -
                "*/status.php*" -
                "*/public.php*" -
                "*/core/img/*" -
                "/index.php/apps/files_sharing/publicpreview.png" -
                "*/*.css" -
                "*/*.js" -
                "/client/*" {                    
                     Disable APM if not already authenticated
                    if { not ( [HTTP::cookie exists MRHSession] && [ACCESS::policy result -sid [HTTP::cookie value MRHSession]] equals "allow" ) } {
                        ACCESS::disable
                    }
                    return          
                }
                "/" {
                     Set the proper sso
                    WEBSSO::select [set foo /Common/CLIENT_INIT_FORMS_SSO]
                    return
                }
                default {}
            }
        }
    }
}

I would probably recommend, if you haven't already, adding additional logging throughout the code to see where the error is happening.

I did add some logging on the HTTP_RESPONSE, and logged the URI and HTTP status code, and the error message come up after that

: Status: 200 http_process_state_prepend - Invalid action:0x109010 (Server side: ...)

I also tried adding logging for the actual payload data, which seemed to have what looked like correct data in it. Here's the extra code I added:

when HTTP_RESPONSE {    
    if {$DEBUG} { log local0. "Uri: $uri" }
    if {$DEBUG} { log local0. "Status: [HTTP::status]" }

     Was returning here because collecting the payload messes up the response to the client (connection reset error). 
    return

     Trigger collection for up to 1MB of data
    if {[HTTP::header exists "Content-Length"] && [HTTP::header "Content-Length"] <= 1048576}{
        set content_length [HTTP::header "Content-Length"]
    } else {
        set content_length 1048576
    }
        if {$DEBUG} { log local0. "Content-Length: $content_length" }

     Check if $content_length is not set to 0
    if { $content_length > 0} {
        HTTP::collect $content_length
    }
}

when HTTP_RESPONSE_DATA {
     do stuff with the payload
    set payload [HTTP::payload]
    if {$DEBUG} { log local0. "Response Payload: '$payload'" }
}

Another thought from looking at similar error messages through devcentral and the SOL's, it seems that non-compliant HTTP responses may cause the issue, and I'm wondering if that's what's going on here. A colleague was saying he tries downloading a 156MB file, and after 155MB, it failed, so he was left with a mostly complete file... This makes me think it might be a content-length mismatch or something that fails at the end of the download...

Is that possible?

Is that possible?

It is possible, but so far we're not sure if this is an LTM or APM related issue. Can you test downloading without the access policy enabled?

You didn't have an HTTP_RESPONSE event before, so I don't think there's any reason to believe the error was happening in the response. Can you add a bunch of logging inside the HTTP_REQUEST event, before and after different evaluations?

That's true. I do get the error regardless of whether the HTTP_RESPONSE handler is in there.

For logging, I have logging on each step (each switch section, and before/after the commands throughout the iRule, and at beginning and end of HTTP_REQUEST. The error occurs after all that each time. And if I have the response in there, it occurs after the HTTP_RESPONSE completes (I also added an HTTP_RESPONSE_RELEASE handler and it occurs after that completes as well)

The URL that I'm going to in order to test this is one of those in which the ACCESS:DISABLE gets executed (a "/public.php" uri). It also happens when the access policy does not get disabled.

Given that, I'm inclined to think it's likely LTM rather than APM.

Given that, I'm inclined to think it's likely LTM rather than APM.

I would tend to agree. So any chance you can limit this to an LTM-only environment to test the download?

Just tried it going straight through the LTM without APM, and all things the same, it gave me the same result.

However, I decided to remove the HTTP profile completely from the VIP, I can download a file without seeing the error in the logs or getting an error prompt in the browser... Tried modifying different settings in the HTTP profile (which inherits from the default http profile with no changes), but can't seem to figure out a way to get it working. If it's on, it fails.

Okay, well then let's do this. Add the following:

when CLIENT_ACCEPTED {
    TCP::collect
}
when CLIENT_DATA {
    log local0. [TCP::payload]
    TCP::release
    TCP::collect
}

Going on the assumption that something is attempting to talk through the VIP with a non-HTTP protocol, you should be able to see this in its raw form directly before the error. You'll also need to disable client side SSL, if possible, for the payload to be readable.