Forum Discussion

yaron123142124_'s avatar
yaron123142124_
Icon for Nimbostratus rankNimbostratus
Dec 03, 2015

Hi how do i remove paramter and random value from url?

how to remove parameter from URL with IRULE F5 but leave all other url syntax unchanged this is the url with f5 ip address

 

http://192.168.1.1/OKFAC_rit/GetOan?order=desc&orokl=time&id=30&identity=564321&client=xml

 

my questions: 1) how do i remove only &identity=564321 parameter and its value(564321 ) form the url?

 

2) i forget to add that i need it to run only for GetOan API URLs regardless of OKFAC_rit prefix. so i need a regular expression that check all the HTTP::REQUEST and check if GetOan exist with both case insensitive and with case sensitive .

 

which mean i need to use [HTTP::request] and NOT [HTTP::query] OR[HTTP::path] .

 

so far i tried: if { [HTTP::request] matches_regex "_GetVideoList_"} { DO SOMETHING HERE} if { [HTTP::request] contains "GetVideoList"} { DO SOMETHING HERE}

 

both didn't helped and it keep skip the IRULE

 

6 Replies

  • Hi,

    I am using the answer-mode as it provides formatting options.

    The TCL
    regsub
    does a substitution in a string based on a search pattern.

    The search pattern is a interpreted as a regular expression.

    (And this makes it more flexible than the
    string map
    solution but may consume more CPU cycles.

    So for a lot of other use cases the solution provided by Stanislas will be the better one.)

    Here is the explanation for the regsub command as used above:
    regsub -nocase -all {(^identity=[^;&]*[;&]?|[;&]identity=[^;&]*)} [HTTP::query] "" new_query
    

    The switches

    -nocase
    and
    -all
    allow to search and replace all matching ranges and handle the search pattern case insensitive. If you are expecting a single instance of the query parameter only and you are sure about upper/lower case characters you can go without both params.

    The search pattern is placed between curly braces

    {...}
    followed by the string to search (in our case the expanded
    [HTTP::query]
    followed by the string which is replacing the found ranges (in our case an empty string
    ""
    ) followed by a variable which is containing the modified string (in our case the new resulting http query represented by the variable
    new_query
    ).

    The search pattern consists actually of two patterns separated by a pipe

    |
    and surrounded by parenthesis
    ()
    .

    This approach allows to search for different search patterns.

    In our case it is required as we may expect the query parameter at the beginning, in the middle of, at the end of or being the only query parameter.

    Lets start with the first pattern of
    ^identity=[^;&]*[;&]?
    :

    The circumflex
    ^
    requires the following pattern to be found at the beginning of the search string (query parameter is the first or first and only parameter). It is followed by a literal string of
    identity=
    which is matching the name of your specific query parameter which has to be replaced. (You may want to replace it i.e. by
    GetOan=
    .) If you want this pattern to be case insensitive you will better write it all lowercase and use the
    -nocase
    switch as described above. The term
    [^;&]*
    now finds all or none directly following characters not matching ";" or "&". In this case the circumflex symbol
    ^
    at the beginning of the class definition negates the search to exclude the known query parameter separators from matching (in regex characters may have different meanings depending on the context they are used). This part of the search pattern will actually find the value of the query parameter (in your case it is the variable part). It will be followed (multiple parameters) or not (single parameters only) by a ";" or "&" separator character. The one instance or not is expressed by the question mark
    ?
    character.

    The second pattern of
    [;&]identity=[^;&]*
    looks for matches if the query parameter is embedded between others or can be found at the end. In both cases we have a leading separator character of ";" or "&" which is indicated by the class of
    [;&]
    (one of the characters between the brackets has two match; please note, that no quantifier like
    *
    or
    ?
    as above is used). Now we try again a literal match for your literal query parameter name including the "=". As in the first search pattern we now have to find the associated parameter value which must not contain ";" or "&" as they are used as separators. This is finally done again by using
    [^;&]*
    . So it will actually find as well a query parameter with an empty value as the
    *
    finds all and nothing.

    Thats it. I wont call me an expert in regex and my terminology may not be 100% accurate.

    To adopt it to another (single and case sensitive) query parameter you will go with:
    regsub {(^GetOan=[^;&]*[;&]?|[;&]GetOan=[^;&]*)} [HTTP::query] "" new_query  
    

    I hope, this answers your question.

    Thanks, Stephan
  • Hi,

    you can try this irule:

    when HTTP_REQUEST {
        set identity [URI::query [HTTP::uri] identity]
        HTTP::uri [string map "&identity=$identity&/&/" [HTTP::uri]]
    }
    
  • Hi,

    i like Stanislas approach (+1) but perhaps we have some more use cases to consider:

    * empty query,

    * parameter at the beginning of, middle of or end of query,

    * relevant parameter being the only parameter in query,

    * query separators might be ampersands or semicolons,

    * relevant parameter might not be case sensitive.

    (I got a similar request from a client a while ago and had to work through it. Please reply if you find the list incomplete or wrong.)

    Here is my approach based on regsub allowing regex and not being case sensitive:
    when HTTP_REQUEST priority 100 {
        if { [HTTP::query] ne "" } {
            regsub -nocase -all {(^identity=[^;&]*[;&]?|[;&]identity=[^;&]*)} [HTTP::query] "" new_query
    
             log local0. "no vars: [regsub -nocase -all {(^identity=[^;&]*[;&]?|[;&]identity=[^;&]*)} [HTTP::query] ""]"
             log local0. "in: [HTTP::uri] | out: [HTTP::path]?$new_query"
    
            if {$new_query ne ""} {
                HTTP::uri [HTTP::path]?$new_query
                 HTTP::respond 200 content "[HTTP::path]?[HTTP::query]\n" noserver
            } else {
                HTTP::uri [HTTP::path]
                 HTTP::respond 200 content "[HTTP::path]\n" noserver
            }
        }
    }
    

    Uncomment lines in the iRule for local testing via cURL:

    curl -v -0 "http://10.131.131.61/mypath?idenTity=0;p1=a;p2=b;identity=123;p3=c&Identity=456"
    * About to connect() to 10.131.131.61 port 80 (0)
    *   Trying 10.131.131.61... connected
    * Connected to 10.131.131.61 (10.131.131.61) port 80 (0)
    > GET /mypath?idenTity=0;p1=a;p2=b;identity=123;p3=c&Identity=456 HTTP/1.0
    > User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 OpenSSL/1.0.1l zlib/1.2.3 libidn/0.6.5
    > Host: 10.131.131.61
    > Accept: */*
    >
    * HTTP 1.0, assume close after body
    < HTTP/1.0 200 OK
    < Connection: close
    < Content-Length: 23
    <
    /mypath?p1=a;p2=b;p3=c
    

    Thanks, Stephan

  • Hi, the [HTTP::request] is probably not the perfect command for your requirements. Or do you really want to parse all request headers as well (i.e. the _GetVideo_List is contained in a cookie name or value)? In case you are expecting the string of "_GetVideoList_" to show up in the request path you may want to use the following (using "string tolower" to be case insensitive): if { [string tolower [HTTP::path]] contains "_getvideolist_" } { } Thanks, Stephan
  • HI i really want to parse all request headers as well . so the answer is -> if { [HTTP::request] contains "_GetVideoList_"} { } OR if { [string tolower [HTTP::request]] contains "_getvideolist_" } { } ?
  • Hi, if the expanded [HTTP::request] does not the search pattern of _getvideolist_ it might be included on the [HTTP::payload]. Do you have access to a capture you can show us (after sanitization of course), please? As I never used the [HTTP::request] command before (always parsed path, query, header names and values separately) I made a quick test on v11.5.1. When sending a POST with some data the data will not be included in the return value of [HTTP::request]. Instead it will show up in [HTTP::payload]. In your sample code above a single test after the lower case conversion (by using the [string tolower ...] should work, in case the pattern is part of the URI, headers or header values. At least it worked fine in v11.5.1. If you want to use a regex, the search pattern can be made case insensitive via preceding (?i): {[HTTP::request] matches_regex {(?i)_getvideo_} } Thanks, Stephan