Modifying multiple entries in a datagroup via api?

Question

We have a datagroup with entries like this:&nbsp;domain1.com := virtual /Common/www.domain1.com_vs_443
domain2.com := virtual /Common/www.domain2.com_vs_443
domain3.com := virtual /Common/www.domain3.com_vs_443 &nbsp;And so forth. This datagroup gets used by an iRule for re-routing traffic based on SNI.&nbsp;I need to be able to make an HTTP call to the F5 to remap these to put up a mainteance page. So in other words I want to modify the above to:&nbsp;domain1.com := virtual /Common/maintenance.domain1.com_vs_443
domain2.com := virtual /Common/maintenance.domain2.com_vs_443
domain3.com := virtual /Common/maintenance.domain3.com_vs_443 &nbsp;I figured out that I can make a curl request as such to delete entries from the datagroup:&nbsp;curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records%20delete%20%7B%20www.domain1.com,www.domain2.com,www.domain3.com%20%7D |jq .&nbsp;And via this article&nbsp;https://community.f5.com/t5/technical-forum/add-new-key-into-data-group-without-updating-entire-list-using/td-p/272699&nbsp;I was able to determine we could add a key using this:&nbsp;curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' 'https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records%20add%20%7b%20www.domain1.com%20%7b%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7d%20%7d' |jq .&nbsp;However then I have to have one HTTP Request per domain. If I need to repoint 20 sites for example I have to make 20 different requests instead of one request with all domains.&nbsp;Is there a way to add multiple records at once? I tried something like this:&nbsp;curl -ku "admin:superS3cret" -X PATCH -H 'Content-type: application/json' -d '{ "name":"dummy" }' 'https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options%3Drecords%20add%20%7B%20www.domain1.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7D%20domain1.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain1.com_vs_443%22%20%7D%20www.domain2.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain2.com_vs_443%22%20%7D%20www.domain3.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain3.com_vs_443%22%20%7D%20www.domain4.com%20%7B%20data%20%22virtual%20%2FCommon%2Fmaintenance.domain4.com_vs_443%22%20%7D%7D' |jq .&nbsp;Here's the URL HTML decoded (so its easier to read):&nbsp;https://lb1.internal.local/mgmt/tm/ltm/data-group/internal/dummy?options=records add { www.domain1.com { data "virtual /Common/maintenance.domain1.com_vs_443" } domain1.com { data "virtual /Common/maintenance.domain1.com_vs_443" } www.domain2.com { data "virtual /Common/maintenance.domain2.com_vs_443" } www.domain3.com { data "virtual /Common/maintenance.domain3.com_vs_443" } www.domain4.com { data "virtual /Common/maintenance.domain4.com_vs_443" }}&nbsp;However I get this error:&nbsp;{
"code": 400,
"message": "one or more properties must be specified",
"errorStack": [],
"apiError": 26214401
}&nbsp;I haven't been able to figure out if adding multiple records isn't supported or if I have the syntax wrong and frankly the documentation here:&nbsp;https://clouddocs.f5.com/api/icontrol-rest/APIRef_tm_ltm_data-group_internal.html&nbsp;and here:&nbsp;https://clouddocs.f5.com/cli/tmsh-reference/v14/modules/ltm/ltm_data-group_internal.html&nbsp;isn't super clear.Frankly the fact that patch doesn't just modify entries sent seems like a bug to me and using ?options record add seems like a hack.Alternatively is there some way to run a modify instead of running delete and then add? That would be even more efficient (less HTTP calls) and less opportunity for a call to fail.&nbsp;I know I can just get all records, modify them and then post all records back. The trouble with that approach is that then I'm touching records I really don't want to touch and having to do some sort of regex replace. This seems like it could have the potential to accidentally modify records I don't want to modify if my regex isn't very explicit. I would prefer a more targeted approach which only modifies the records that need modifying and that touches nothing else.Thanks

sebastiansierra · Accepted Answer

Hi&nbsp;Brad_Baker
You can additionally edit the content for a specific register
curl -ku "admin:Logi-123" -X PATCH -H 'Content-type: application/json' -d '{ "name":"test_domain" }' https://localhost/mgmt/tm/ltm/data-group/internal/test_domain?options=records%20modify%20%7B%20key3%20%7B%20data%20ke3new%20%7D%20%7D|jq .
as tmos you can use all of this option to interact with the data-groups
add      delete      modify     none        replace-all-with
&nbsp;to avoid mistakes use a tool to encode and decode the URL.&nbsp;

brad_baker · Answer

Perfect - that's exactly what I was trying to do! I was just using add instead of modify - it seems that was the issue.

Thank you so much figuring that out! 🙂 Now I don't have to worry about accidentally messing up any other datagroup records during an update.

sebastiansierra · Answer

Hi&nbsp;Brad_Baker,
The easiest way is to delete all the content of the datagroup and then add the new records, for delete all the records use this curl:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.10/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{ "records": []}'
To add the new records in only one call you can use this curl:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{"records": [{"name": "domain1.com","data": "virtual /Common/maintenance.domain1.com_vs_443"},{"name": "domain2.com","data": "virtual /Common/maintenance.domain2.com_vs_443"},{"name": "domain3.com","data": "virtual /Common/maintenance.domain3.com_vs_443"}]}'
Hope it works.
&nbsp;
&nbsp;

sebastiansierra · Answer

Hi Brad_Baker,
to avoid mistakes you can use this curl call format to have more clear the new records that you want to create:
curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' \
--header 'Content-Type: application/json' \
--data '{
    "records": [
        {
            "name": "domain1.com",
            "data": "virtual /Common/maintenance.domain1.com_vs_443"
        },
        {
            "name": "domain2.com",
            "data": "virtual /Common/maintenance.domain2.com_vs_443"
        },
        {
            "name": "domain3.com",
            "data": "virtual /Common/maintenance.domain3.com_vs_443"
        }
    ]
}'

brad_baker · Answer

Hey&nbsp;Sebastiansierra&nbsp;thanks for the response and suggestions.Unfortunately, the recommendation above won't accomplish what I am trying to do.I have approximately 100-150 entries in my datagroup. When I run the command recommended above:&nbsp;curl -ku "admin:admin" -X PATCH 'https://192.168.30.1/mgmt/tm/ltm/data-group/internal/test_domain' --header 'Content-Type: application/json' --data '{"records": [{"name": "domain1.com","data": "virtual /Common/maintenance.domain1.com_vs_443"},{"name": "domain2.com","data": "virtual /Common/maintenance.domain2.com_vs_443"},{"name": "domain3.com","data": "virtual /Common/maintenance.domain3.com_vs_443"}]}'It replaces all 100-150 entries with the 4 entries in the data section.I don't want to delete existing entries; I only want to edit specific records. So I'm looking to do a targeted modification to only touch the records that need changing and leave the rest alone.Also, I understand I could do a GET to get all records, use a regex to replace just what I want to replace, and then do a PUT/PATCH to send the modified version back.But that hinges on getting a regex exactly right. If my regex matches wrong (i.e. too much) I could accidentally edit the wrong record. It just seems a bit risky to do it using regex/pattern matching. Thus wanting a targeted approach that edits specific records.Is that possible or does the F5 api just not support that?

brad_baker · Answer

Sebastiansierra&nbsp;manually add domain4.com, domain5.com and domain6.com into the datagroup from the UI. Then run the curl request above. You'll see domain4/5/6 disappear. That's what I want to prevent. I want the query to only touch domain1-3 and leave domain4-6 alone.&nbsp;

Forum Discussion

Modifying multiple entries in a datagroup via api?

9 Replies

Recent Discussions

BIG-IP DNS: Check Status Of Multiple Monitors Against Pool Member

Open Redirection Mitigation

F5Access | MacOS Sonoma

enable tls1.2 on management interface on F5 ltm running version 10.x

[ASM] - what is "Browser Challenge file" ?

Related Content

CSV to Address External Datagroup File

iRule to modify a content-security-policy header

Help me! About datagroup

Modifying iCall from TMSH

F5 Distributed Cloud – Multiple custom certificates for HTTP/TCP LB