Forum Discussion

Chris_Phillips's avatar
Chris_Phillips
Icon for Nimbostratus rankNimbostratus
Sep 24, 2013

Merging multiple classes

Howdy,

 

I've an irule which is referencing a list of URI's stored in a class, doing a starts_with match on it.

 

is there any way to take multiple classes and merge them dynamically (probably on CLIENT_ACCEPTED) into a single list upon which the same single match can be performed? And if there is, would this be horribly inefficient compared to using a "proper" class or would, after the new class is made, there realistically be no difference?

 

Thanks

 

7 Replies

  • I´m not aware of being able to merge classes on the fly. That´s why it might be the easiest approach just to craft a list of classes (perhaps you are using names for your classes based on a decission criteria, i.e. 1st or 2nd label of the client IP address).

     

    Now you will use a for loop to run your 'class match -value' for each member of the list. So each data group will be looked up.

     

    Trying to dump each data group per client request and merging them will be probably too resource consuming as you already pointed out.

     

  • One important thing to add I just stumbled across on the weekend:

     

    In v11.4 you will always need a key-value pair in your data group entries.

     

    (AskF5 SOL14671)

     

    A data group with keys only will load, but cannot be modified afterwards ('general database error').

     

    You will need to add a value per key - i.e. '1' - even if your are not using it.

     

  • I thought about looping through them... I currently do this elsewhere in the same rule for IP matching, but as I'm doing a starts_with on a URI, I need to get out the longest match of all three or four lists, which isn't possible from looping through them, without some manual string length checks outside of this, which mostly sounds too cumbersome to bother with for the operational benefit of being able to logically separate out different parts of a website into different classes.

     

  • Just a few additional thoughts:

     

    1. Probably the easiest thing would be, as Stephen relates, to key the data groups so that 1) a specific subset of data is kept in a specific data group, and b) the iRule knows which data group to poll based on that key.

       

    2. It would technically be possible, albeit CPU-consuming, to dump the contents of each data group into a single list object via [class get ] and then perform list search functions.

       

    3. Depending on the size of the data groups (ie. thousands of records each), it may also make sense to have an asynchronous management plane process to regularly aggregate the data into a single separate data group.

       

    A data group shouldn't change so often that you would need to build a new aggregate list on each client request or connection.

     

  • How about looking for a specific string in the URI via regex pattern and to use 'switch' and a jump list to lookup a specific datagroup only?

     

  • But a list can't be used with a single match operation right? I'm certainly not currently interested in reworking all that logic, so that'd be a blocker.

     

    The lists are groups of URI's to which we are restricting access to certain IP ranges. Currently there's only a single list, and if we want to block a specific page, we update the URI list. If we can use multiple classes, we can update the config at more of an object level rather than individual class entries. So I have a field in another dgl called "uri_dgl" and I could then go from having [code]"uri_dgl" { "permanent_list" }[/code] to [code]"uri_dgl" { "permanent_list temporary_list" }[/code] rather than adding a handful of entries to the "permanent_list" class.

     

    As for size, it's only ever up to say 20 max, and that'd be during an upgrade window. As we sit behind a CDN, our TCP connection count is vastly lower than out HTTP_REQUEST count, so I'd be happy using the CLIENT_ACCEPTED as the periodic cleanup / build mechanism. But is it's just not effifient a thing to do overall, there's clearly no point.

     

    it's not really that important, just a nice to have that would make the solution I've architected easier for our support guys to understand and maintain.

     

  • But a list can't be used with a single match operation right?

    I'd argue that a list is actually more flexible than a data group:

    http://www.tcl.tk/man/tcl8.4/TclCmd/lsearch.htm

    So say you had a few simple single-value data groups. Example:

    dgl_test_1

    "/test1" := ""
    "/test2" := ""
    "/test3" := ""
    "/test4" := ""
    

    dgl_test_2

    "/foo1" := ""
    "/foo2" := ""
    "/foo3" := ""
    "/foo4" := ""
    

    dgl_test_3

    "/bar1" := ""
    "/bar2" := ""
    "/bar3" := ""
    "/bar4" := ""
    

    And an iRule that did something like this:

    when CLIENT_ACCEPTED {
        set uri_list "[class get dgl_test_1] [class get dgl_test_2] [class get dgl_test_3]"
    }
    

    ** notice the space between the [class get ] commands.

    This would give you a single list of URIs that you could sort or search.