BIG-IP Next Automation: AS3 Basics
I need a little Mr. Miyagi right now to grab my face and intently look me in the eye and give me a "Concentrate! Focus power!" For those of you youngins' who don't know who that is, he's the OG Karate Kid mentor. Anyway, I have a thousand things I want to say about AS3 but in this article, I'll attempt to cut this down to a narrow BIG-IP Next-specific context to get you started. It helps that last December I did a five-part streaming series on AS3 in the BIG-IP classic context. If you haven't seen that, you have my blessing to stop right now, take some time to digest AS3 conceptually and practice against workloads and configurations in BIG-IP classic that you know and understand, before returning here to embrace all the newness of BIG-IP Next. AS3 is FOUNDATIONAL in BIG-IP Next In classic BIG-IP, you could edit the bigip.conf file directly, use tmsh commands, or iControlREST commands to imperatively create/modify/delete BIG-IP objects. With the exception of system configuration and shared configuration objects, this is not the case with BIG-IP Next. All application configuration is AS3 at its lowest state level. This doesn't mean you have to work primarily in AS3 configuration. If you utilize the migration utility in Central Manager, it will generate the AS3 necessary to get your apps up and running. Another option is to use the built-in http FAST template (we'll cover FAST in later articles) to build out an application from scratch in the GUI. But if you use features outside the purview of that template, or you need to edit your migration output, you'll need to work in the AS3 configuration declaration, even if just a little bit. Apples to Apples It's a fun card game, no? My family takes it to snarky absurd levels of sarcasm, to the point that when we play with "outsiders" we get lots of blank looks and stares as we're all rolling on the floor laughing. Oh well, to each his own. But we're here to talk about AS3, right? Well, in BIG-IP Next, there is a compatibility API for AS3, such that you can take a declaration from BIG-IP classic and as long as the features within that declaration are supported, it should "just work" via the Central Manager API. That's pretty cool, right? Let's start with a basic application declaration from the recent video posted by Mark_Dittmerexploring the API differences between classic and Next. { "class": "ADC", "schemaVersion": "3.0.0", "id": "generated-for-testing", "Tenant_1": { "class": "Tenant", "App_1": { "class": "Application", "Service_1": { "class": "Service_HTTP", "virtualAddresses": [ "10.0.0.1" ], "virtualPort": 80, "pool": "Pool_1" }, "Pool_1": { "class": "Pool", "members": [ { "servicePort": 80, "serverAddresses": [ "10.1.0.1", "10.1.0.2" ] } ] } } } } A simple VIP with a pool with two pool members. A toy config to be sure, but it is useful here to show the format (JSON) of an AS3 declaration and some of the schema as well. With the compatibility API, this same declaration can be posted to a classic BIG-IP like this: POST https://<BIG-IP IP Address>/mgmt/shared/appsvcs/declare Or a BIG-IP Next instance like this: POST https://<Central Manager IP Address>/api/v1/spaces/default/appsvcs/declare?target_address=<BIG-IP Next instance IP Address> For those already embracing AS3, this compatibility API in BIG-IP Next should make the transition easier. AS3 Workflow in BIG-IP Next With BIG-IP classic, you had to install the AS3 package (technically an iControl LX, or sometimes referenced as an iApps v2 package) onto each BIG-IP system you wanted to use the AS3 declarative configuration model on. Each BIG-IP was an island, and the configuration management of the overall system of BIG-IPs was reliant on an external system for source of truth. With BIG-IP Next, the Central Manager API has native AS3 support so there are no packages to install to prepare the environment. Also, Central Manager is the centralized AS3 interface for all Next instances. This has several benefits: A singular and centralized source of truth for your configuration management No external package management requirements Tremendous improvement in API performance management since most of the heavy lifting is offloaded from the instances and onto Central Manager and the control-plane functionality that remains on the instance is intentionally designed for API-first operations The general application deployment workflow introduced exclusively for Next, which I'll reference as the documents API, is twofold: Create an application service First, you create the application service on Central Manager. You can use the same JSON declaration from the section above here, only the API endpoint is different: POST https://<Central Manager IP Address>/api/v1/spaces/default/appsvcs/documents A successful transaction will result in an application service document on Central Manager. A couple notes on this at time of writing: Documents created through the API are not validated against the journeys migration tool that is available for use in the Central Manager GUI. Documents are not schema validated at the attribute level of classes, so whereas a class used in classic might be supported in Next, some of the attributes might not be. This means that whereas the document creation process can appear successful, the deployment will fail if classes and/or class attributes supported in classic BIG-IP are present in the AS3 declarations when an attempt to apply to an instance occurs. Deploy the application service Assuming, however, all your AS3 work is accurate to the Next-supported schema, you post the specified document by ID to the target BIG-IP Next instance, here as a JSON payload versus a query parameter on the compatibility API shown earlier. POST https://<Central Manager IP Address>/api/v1/spaces/default/appsvcs/documents/<Document ID>/deployments { "target": "<BIG-IP Next Instance IP Address>" } At this point, your service should be available to receive traffic on the instance it was deployed on. Next Up... Now that we have the theory in place, join me next time where we'll take a look at working with a couple application services through both approaches. Resources CM App Services Management AS3 Schema AS3 User Guide (classic, but useful) AS3 Reference Guide (classic, but useful) AS3 Foundations (streaming series)BIG-IP Next Automation: Working with the AS3 API endpoints
In my last article I covered the basics of AS3 as it relates to getting started with automation with BIG-IP Next. I also walked through an application migration in a previous article that addresses some of the issues you'll need to work through moving to Next, but whereas I touched the AS3 slightly in the workflow, all the work was accomplished in the Central Manager web UI. In this article, I'll walk you through creating two applications, one a simple DNS load balancing application and the other a TLS-protected HTTP application with an associated iRule. For each application, I'll use the compatibility API and the documents API for working through the CRUD operations. Creating the declarations You can go about this a few different ways. You can start from the AS3 schema reference and climb up from scratch, you can spin up Visual Studio Code and work with the F5 Extension to interrogate your own BIG-IP configurations and use the AS3 Config Converter to automagically do the work for you, or you can just ask chatGPT to generate the AS3 for you to get started like I did. And after that didn't work without a lot of tweaking...I went back to VSCode. Example 1 - DNS application service declaration Here's what I ended up with for the DNS application service: { "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/latest/as3-schema.json", "class": "AS3", "declaration": { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:3a71dceb-f56c-4dc1-901a-2feae0244c46", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "Common": { "class": "Tenant", "Shared": { "class": "Application", "template": "shared", "vip.ns-cluster-1": { "layer4": "udp", "pool": "pool.ns-cluster-1", "translateServerAddress": true, "translateServerPort": true, "class": "Service_UDP", "profileUDP": { "bigip": "/Common/udp" }, "virtualAddresses": [ "10.100.100.100" ], "virtualPort": 53, "snat": "auto" }, "pool.ns-cluster-1": { "members": [ { "addressDiscovery": "static", "servicePort": 53, "serverAddresses": [ "10.10.100.101", "10.10.100.102", "10.10.100.103", "10.10.100.104" ], "shareNodes": true } ], "monitors": [ { "bigip": "/Common/udp" } ], "class": "Pool" } } } } } Note that in BIG-IP Next, there isn't an alternative to the AS3 class, so that wrapper for the ADC class declaration is unnecessary and will result in an error if posted. So the only change required at this time is to remove the wrapper, and change common/shared to tenant1/dnsapp1 as shown below. { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:3a71dceb-f56c-4dc1-901a-2feae0244c46", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "tenant1": { "class": "Tenant", "dnsapp1": { "class": "Application", "template": "shared", "vip.ns-cluster-1": { "layer4": "udp", "pool": "pool.ns-cluster-1", "translateServerAddress": true, "translateServerPort": true, "class": "Service_UDP", "profileUDP": { "bigip": "/Common/udp" }, "virtualAddresses": [ "10.100.100.100" ], "virtualPort": 53, "snat": "auto" }, "pool.ns-cluster-1": { "members": [ { "addressDiscovery": "static", "servicePort": 53, "serverAddresses": [ "10.10.100.101", "10.10.100.102", "10.10.100.103", "10.10.100.104" ], "shareNodes": true } ], "monitors": [ { "bigip": "/Common/udp" } ], "class": "Pool" } } } } But wait! There's more!Now that I'm channeling my inner Billy Mays, the declaration is not quite ready for Next. After a quick test or five or six, there are some problems with my schema in the move to Next. Here are the necessary changes, followed by the final declaration I'll used with the API endpoints. Swapped out the UDP monitor for ICMP since there is not currently a UDP monitor available Removed the profileUDP, layer4, and translateServerPort attributes from the Service_UDP class { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:3a71dceb-f56c-4dc1-901a-2feae0244c46", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "tenant1": { "class": "Tenant", "dnsapp1": { "class": "Application", "template": "shared", "vip.ns-cluster-1": { "pool": "pool.ns-cluster-1", "translateServerAddress": true, "class": "Service_UDP", "virtualAddresses": [ "10.100.100.100" ], "virtualPort": 53, "snat": "auto" }, "pool.ns-cluster-1": { "members": [ { "addressDiscovery": "static", "servicePort": 53, "serverAddresses": [ "10.10.100.101", "10.10.100.102", "10.10.100.103", "10.10.100.104" ], "shareNodes": true } ], "monitors": [ "icmp" ], "class": "Pool" } } } } Example 2 - TLS-protected HTTP application service with iRule declaration And here's the HTTP application service as converted in VSCode but without the AS3 class wrapper: { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:bd9c9728-8c20-4c4d-a625-68450e35e133", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "Common": { "class": "Tenant", "Shared": { "class": "Application", "template": "shared", "vip.acme_labs": { "layer4": "tcp", "pool": "pool.acme_labs", "iRules": [ { "use": "/Common/Shared/full_uri_decode" } ], "translateServerAddress": true, "translateServerPort": true, "class": "Service_HTTPS", "serverTLS": "/Common/Shared/cssl.acme_labs", "profileHTTP": { "bigip": "/Common/http" }, "profileTCP": { "bigip": "/Common/tcp" }, "redirect80": false, "virtualAddresses": [ "172.16.101.133" ], "virtualPort": 443, "snat": "auto" }, "pool.acme_labs": { "loadBalancingMode": "least-connections-member", "members": [ { "addressDiscovery": "static", "servicePort": 80, "serverAddresses": [ "172.16.102.5" ], "shareNodes": true } ], "monitors": [ { "bigip": "/Common/http" } ], "class": "Pool" }, "www.acmelabs.com": { "class": "Certificate", "certificate": { "bigip": "/Common/www.acmelabs.com" }, "privateKey": { "bigip": "/Common/www.acmelabs.com" } }, "cssl.acme_labs": { "certificates": [ { "certificate": "/Common/Shared/www.acmelabs.com" } ], "class": "TLS_Server", "tls1_0Enabled": true, "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": false, "singleUseDhEnabled": false, "insertEmptyFragmentsEnabled": true }, "full_uri_decode": { "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICMgZGVjb2RlIG9yaWdpbmFsIFVSSS4KICBzZXQgdG1wVXJpIFtIVFRQOjp1cmldCiAgc2V0IHVyaSBbVVJJOjpkZWNvZGUgJHRtcFVyaV0KICAjIHJlcGVhdCBkZWNvZGluZyB1bnRpbCB0aGUgZGVjb2RlZCB2ZXJzaW9uIGVxdWFscyB0aGUgcHJldmlvdXMgdmFsdWUuCiAgd2hpbGUgeyAkdXJpIG5lICR0bXBVcmkgfSB7CiAgICBzZXQgdG1wVXJpICR1cmkKICAgIHNldCB1cmkgW1VSSTo6ZGVjb2RlICR0bXBVcmldCiAgfQogIEhUVFA6OnVyaSAkdXJpCiAgbG9nIGxvY2FsMC4gIk9yaWdpbmFsIFVSSTogW0hUVFA6OnVyaV0iCiAgbG9nIGxvY2FsMC4gIkZ1bGx5IGRlY29kZWQgVVJJOiAkdXJpIgp9" } } } } } This is mostly ok with the exception of the certificate handling in lines 56-70. If I was posting this back to my local BIG-IP in place of the imperative configuration it'd be fine. But Central Manager has no context for where those certificates are so I'll need to do a little work here to prep the declaration. I need to drop the certificate and key into the Certificate class (your security-sense should be tingling, remember these are private keys so in your environment you'd be pulling these credentials in from a vault and NOT storing these in a file) and then updating the reference to the local object in the TLS_Server class. NOTE: It might be confusing for long-time BIG-IP users, but the TLS_Server class in AS3 is the equivalent of a client-ssl profile, and the TLS_CLIENT class in AS3 is the equivalent of a server-ssl profile. This change was made in AS3 to align more with industry-standard nomenclature. After these changes, and changes to Common/Shared, the updated declaration is shown below. { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:bd9c9728-8c20-4c4d-a625-68450e35e133", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "tenant2": { "class": "Tenant", "httpsapp1": { "class": "Application", "template": "shared", "vip.acme_labs": { "layer4": "tcp", "pool": "pool.acme_labs", "iRules": [ { "use": "/Common/Shared/full_uri_decode" } ], "translateServerAddress": true, "translateServerPort": true, "class": "Service_HTTPS", "serverTLS": "/Common/Shared/cssl.acme_labs", "profileHTTP": { "bigip": "/Common/http" }, "profileTCP": { "bigip": "/Common/tcp" }, "redirect80": false, "virtualAddresses": [ "172.16.101.133" ], "virtualPort": 443, "snat": "auto" }, "pool.acme_labs": { "loadBalancingMode": "least-connections-member", "members": [ { "addressDiscovery": "static", "servicePort": 80, "serverAddresses": [ "172.16.102.5" ], "shareNodes": true } ], "monitors": [ { "bigip": "/Common/http" } ], "class": "Pool" }, "www.acmelabs.com": { "class": "Certificate", "certificate": "-----BEGIN CERTIFICATE-----\nMIIHQTCCBimgAwIBAgIQFxO0vIztEEcAAAAAUQF6LjANBgkqhkiG9w0BAQsFADCBujELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUVudHJ1c3QsIEluYy4xKDAmBgNVBAsTH1NlZSB3d3cuZW50cnVzdC5uZXQvbGVnYWwtdGVybXMxOTA3BgNVBAsTMChjKSAyMDEyIEVudHJ1c3QsIEluYy4gLSBmb3IgYXV0aG9yaXplZCB1c2Ugb25seTEuMCwGA1UEAxMlRW50cnVzdCBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eSAtIEwxSzAeFw0yMDAzMjYyMTExNTZaFw0yMjAzMTQyMTQxNTVaMGoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpXYXNoaW5ndG9uMRAwDgYDVQQHEwdTZWF0dGxlMRowGAYDVQQKExFGNSBOZXR3b3JrcywgSW5jLjEYMBYGA1UEAwwPKi5lbWVhLmY1c2UuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAt6FDfpu8jBbE8dew0m5t2ax/p6LE0mI0BMJIZA1TxglDvQjgVethDPWp7rTr655ZNuYUZ4p/QV/Uummo0NxhE4VQyIK1tcKnGs/tX2BVmx/augrcqOpGwZKAeKsRDxB8UBS/BmovlQQgRqBym3lg7AewI20BwtSvrCSviGmByBPW7cjOFoe8n706XZvEDFiZgj/OuV2V1giCzqqKUJ5mLAqSh25465IVcTJQxKkok668rHOgpUO2GDav7cnrtLm71Oxv6m64gcQJ+e2xzaxa0/OfykuXn4W84RFKwm6im3lAbgNI+CwCjTNtXXs88TxMG49GuTol9ddeS+4aF9GvCQIDAQABo4IDkDCCA4wwKQYDVR0RBCIwIIIPKi5lbWVhLmY1c2UuY29tgg1lbWVhLmY1c2UuY29tMIIB9wYKKwYBBAHWeQIEAgSCAecEggHjAeEAdwBVgdTCFpA2AUrqC5tXPFPwwOQ4eHAlCBcvo6odBxPTDAAAAXEYy223AAAEAwBIMEYCIQDAvv+hvpE9l0BnPH3ouvKJOyTTrLNRK6qZiHrEm9G3iAIhAIlqyaByyF2OHUAqNnfk7DalviCjaHPzqEmYnsrMIXV9AHYAh3W/51l8+IxDmV+9827/Vo1HVjb/SrVgwbTq/16ggw8AAAFxGMttuQAABAMARzBFAiEAnH87ThX2oxA89e1wDaslF8zZrbu/OG8Jx3I7zqVAtkACIB90UYajoUjMoqTP36sb/tU6N776FNsflbScLedtiqPSAHcAVhQGmi/XwuzT9eG9RLI+x0Z2ubyZEVzA75SYVdaJ0N0AAAFxGMtt3wAABAMASDBGAiEAh6gVTPW97krycFbcH9OcLu/lTRSkfeCbMqUYBXlCtKICIQCmGMSIJNZYFIM3mTD0hb2VDGOMCjHkAE5hiJ5VuLEgswB1ALvZ37wfinG1k5Qjl6qSe0c4V5UKq1LoGpCWZDaOHtGFAAABcRjLbbQAAAQDAEYwRAIgdBV5qHR7nM97nmvdlSK3QLcsq+cr6qd+xns+9Wbv1pcCIBMdw4C5iEMKpwdyLRDR86jQC2v8op/klavXFfYGZ9QyMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUHAwIwMwYDVR0fBCwwKjAooCagJIYiaHR0cDovL2NybC5lbnRydXN0Lm5ldC9sZXZlbDFrLmNybDBLBgNVHSAERDBCMDYGCmCGSAGG+mwKAQUwKDAmBggrBgEFBQcCARYaaHR0cDovL3d3dy5lbnRydXN0Lm5ldC9ycGEwCAYGZ4EMAQICMGgGCCsGAQUFBwEBBFwwWjAjBggrBgEFBQcwAYYXaHR0cDovL29jc3AuZW50cnVzdC5uZXQwMwYIKwYBBQUHMAKGJ2h0dHA6Ly9haWEuZW50cnVzdC5uZXQvbDFrLWNoYWluMjU2LmNlcjAfBgNVHSMEGDAWgBSConB03bxTP8971PfNf6dgxgpMvzAdBgNVHQ4EFgQUaEk3Dl8YuTsNPJ0vhVIKTKZfnNIwCQYDVR0TBAIwADANBgkqhkiG9w0BAQsFAAOCAQEAD8DmFFgnU2veCzDyeoF12bbZfF9oA3nOTY7z2WjYy7/5hyKg6FXKwkXVji13g6RNFVQ03mqcXTN8/AhnHz7dnhWF39WhdH08suWLQrmIT2dPBKTF1aQcURIpOddemsZMx6NCFjgcAHLcK/nPDPsfMXq5tRXInjPyGd38TooIeAfGGPiTrgL3UU8ByQPxriOf4V5i66BOWH8wDViPBeXaDSdgcXhrDXAAt/nArVmI7orK+t/0iCzoeg9pGH39+/G1VansfbTcBbKnqVCxDplUiCXLlD17mN45n9estajf4tnpiXkqBIC14o742HAeqpV9T9wzUbJFo5BWMtpHtPZu2A==\n-----END CERTIFICATE-----", "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEowIBAAKCAQEAt6FDfpu8jBbE8dew0m5t2ax/p6LE0mI0BMJIZA1TxglDvQjgVethDPWp7rTr655ZNuYUZ4p/QV/Uummo0NxhE4VQyIK1tcKnGs/tX2BVmx/augrcqOpGwZKAeKsRDxB8UBS/BmovlQQgRqBym3lg7AewI20BwtSvrCSviGmByBPW7cjOFoe8n706XZvEDFiZgj/OuV2V1giCzqqKUJ5mLAqSh25465IVcTJQxKkok668rHOgpUO2GDav7cnrtLm71Oxv6m64gcQJ+e2xzaxa0/OfykuXn4W84RFKwm6im3lAbgNI+CwCjTNtXXs88TxMG49GuTol9ddeS+4aF9GvCQIDAQABAoIBAGeJbdz9QppaXEFgNDryOM37DR8gD4nwBRSJ1vdS7GFE6AS19Id9aAM+oMoPCNaZOgRSRj77QDVEK1XQLXdWSwYOrTXhPUN2tXHQuy6DysDkfRdY+IHlVm/egsGG8t9jlDQy/mJHjPygjvJDlVtEXPm4e//9fni0IzkUlkR7+MkuMT3vvKGYnUNTlI1hJokcNJ75r91O82j+qQsmvJG3FOUn0DpnEBgIvEbFvD3wMHY1K/fTUsBVJMKkjjXmjykGB9y7V4oKHQLsxH+lrUneWdD/s23hoVgAV31YeXtf7mI/eWPJt6DiGwTfaNcNcptvwugsR/7jCWaKS9Hya/qbJuECgYEA6wNm2doCmwl6ksbKSik0VgVha7DN3VjoFTDFcYZNfjB/kr6/xSODbAxJMwQdevFj2eWQxeHZnJc96x2xWzCA3mp1BhNzcfT8XRZ4LMcLUpcl1VXUEVc562vL8AdVuSODDc/rBXP/aFSXGdE+ZSPhYBrNlnK1FY10aaGsrEaRxL8CgYEAyAcyKVKY+wpdweiVXsUHIHAwQUmi8pKd1j5KlCjJkn2Wtiqex0v1eDy2/iKrZDWRiRFE4WOIb7A9GYm7FfqDyn9WvVNI0bz8Ywi+bCTdawGZ8H328q3R4/xIPprGmKV6olQHHUGZUNkLTK+cDHK4w9JRSf9kB6PUgGnBTgZoFjcCgYB/E2bQ03ZnOLfjl8QYV7Fp9hzYa1DVqFZN5wJMQW+zlSvWQHhXc72Ddh06jbYXHWF9mAkxRs8xQgKEGJknEtIL8gp3D5tz+iFfgF/Y7oPr07jsYy15du3lo3MxxfWPV2ls1YlieHeZhWvy1NblP4KFQdj6yemqzsMsvvQsbzgw5wKBgQCASZ0yQ3c6Cnv3UWP7VAIuG8XXGZMYYFA6h9jtDPu6qDFwxATxbRYR916lvzaNHo4oiprSszNd7npBVsRWZEUCKolHA5NAcSStn34BfeNELdK9Gwy2uCRVRAhRnpKgdAEi+yFU8i2SXKGSnU5H7Yvyi4D3JITTIY+4jBseH53CIQKBgByjoPYp+eMXpUmg4W5M1irXGm8sjrRBKvnxu9L+etvajWIb+AUAtoNoQmcKpf8bBK84PdCwiDSQmRDbWieT9RsSqbyWOcQf2C2L0qujUb+bM+kSTYp4oAV/rukoZ46NHjYBE3NbI7HcspWbpu5zl0Ke9pLvDwFwrmRy5KM7EiSh\n-----END RSA PRIVATE KEY-----" }, "cssl.acme_labs": { "certificates": [ { "certificate": "www.acmelabs.com" } ], "class": "TLS_Server", "tls1_0Enabled": true, "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": false, "singleUseDhEnabled": false, "insertEmptyFragmentsEnabled": true }, "full_uri_decode": { "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICMgZGVjb2RlIG9yaWdpbmFsIFVSSS4KICBzZXQgdG1wVXJpIFtIVFRQOjp1cmldCiAgc2V0IHVyaSBbVVJJOjpkZWNvZGUgJHRtcFVyaV0KICAjIHJlcGVhdCBkZWNvZGluZyB1bnRpbCB0aGUgZGVjb2RlZCB2ZXJzaW9uIGVxdWFscyB0aGUgcHJldmlvdXMgdmFsdWUuCiAgd2hpbGUgeyAkdXJpIG5lICR0bXBVcmkgfSB7CiAgICBzZXQgdG1wVXJpICR1cmkKICAgIHNldCB1cmkgW1VSSTo6ZGVjb2RlICR0bXBVcmldCiAgfQogIEhUVFA6OnVyaSAkdXJpCiAgbG9nIGxvY2FsMC4gIk9yaWdpbmFsIFVSSTogW0hUVFA6OnVyaV0iCiAgbG9nIGxvY2FsMC4gIkZ1bGx5IGRlY29kZWQgVVJJOiAkdXJpIgp9" } } } } } I didn't mention it above, but you'll notice that the iRule is base64 encoded. The conversion to AS3 in VSCode did that automatically. You can do the same for the certificate and privateKey attributes as well if you want, but that'll need the base64 attribute within the curly brackets like the iRule. Billy Mays here again...buy 1, get another free!Like the DNS app, there are a few things native to classic in this declaration that aren't supported in Next, so we need to make a few more changes after a few tests: I removed profileHTTP and profileTCP attributes from the Service_HTTPS class. These are allowed, but since I am not setting anything non-default, I don't need them. As is, they were not acceptable referencing bigip classic profiles Removed layer4 and translateServerPort attributes from the Service_HTTPS class as they are not currently supported in Next Removed tls1_Enabled, singleUseDhEnabled, and insertEmptyFragmentsEnabled attributes from TLS_Server class as they are not currently supported in Next. Added the ciphers attribute with RSA value to the TLS_Server class. The instance would not accept the deployment without this, I got an expired or invalid certificate error without it. Changed the iRules refererence in the Service_HTTPS class from a classic BIG-IP object to a local declaration object. These final changes resulted in the following declaration I'll use with the API endpoints: { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:bd9c9728-8c20-4c4d-a625-68450e35e133", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "tenant2": { "class": "Tenant", "httpsapp1": { "class": "Application", "template": "shared", "vip.acme_labs": { "pool": "pool.acme_labs", "iRules": [ "full_uri_decode" ], "translateServerAddress": true, "class": "Service_HTTPS", "serverTLS": "cssl.acme_labs", "redirect80": false, "virtualAddresses": [ "172.16.101.133" ], "virtualPort": 443, "snat": "auto" }, "pool.acme_labs": { "loadBalancingMode": "least-connections-member", "members": [ { "addressDiscovery": "static", "servicePort": 80, "serverAddresses": [ "172.16.102.5" ], "shareNodes": true } ], "monitors": [ "http" ], "class": "Pool" }, "www.acmelabs.com": { "class": "Certificate", "certificate": "-----BEGIN CERTIFICATE-----\nMIIC3DCCAcSgAwIBAgIGAZAW7PncMA0GCSqGSIb3DQEBDQUAMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDAeFw0yNDA2MTQxMzI1NDdaFw0zNDA2MTIxMzI1NDdaMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMYpeRm4f1mPgW7STMM4gZXZ5p02nCWshNwVkaOLpRJAOdR2ZpuhLW4tWpAssvmTRlS0cFjZKA6ecVg4Q7+wvw7dIG8gVAviOqmHb6sDaomBTn3+ISFYW0Uxb1GNvZqlktJQI7hCsaS5Kf/f4pImVa8jQffWTdgLwxCm+0suaXy1XykVOCdOs1lsCOHjMoVREWxLIAtzMpqdO+8IRhSJgPJPf3GnY861T0LDjuT5rgwY1qK/H2NuEcPWOWVtqTN9aQAz9cKxDbJq48U8adzrl6G8uUYlEPEtneePErygy8wRk8KkVNkuDj5gQKxi3b3Q8/K7bPhh9aUnZRQWmhVTw2kCAwEAATANBgkqhkiG9w0BAQ0FAAOCAQEAOh3doWxnjb5j5XojnEtYUWJG6yw9a3xZhEiq7myWz7apmy5eAe0QAL9kFAuiBwgjqwzPCXzMDp21FdLC+o9Znx5A8kXE2W2G+h36kc21f3v0jumRdkU1zZ9py9iKHAOUSAYsALNWH4mosFFbodpqcFZL7Fqmh/AoIcqY3GqSWOZ6geYbMIOwTZFnsuE1LTjJrnypz1ZyglGoftzU9j501aq3eJ3YUyRIZ28/ARJxn4sUfdvjvs31EdFEOOC6hwN2U7JXdWWK/fATTenglSkUqChJRW6kRL7uFf6FCCZjXyGINJnOYVz+8gxDWA557+ogYfEquQVML5gvMK9Ff67W6A==\n-----END CERTIFICATE-----", "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAxil5Gbh/WY+BbtJMwziBldnmnTacJayE3BWRo4ulEkA51HZmm6Etbi1akCyy+ZNGVLRwWNkoDp5xWDhDv7C/Dt0gbyBUC+I6qYdvqwNqiYFOff4hIVhbRTFvUY29mqWS0lAjuEKxpLkp/9/ikiZVryNB99ZN2AvDEKb7Sy5pfLVfKRU4J06zWWwI4eMyhVERbEsgC3Mymp077whGFImA8k9/cadjzrVPQsOO5PmuDBjWor8fY24Rw9Y5ZW2pM31pADP1wrENsmrjxTxp3OuXoby5RiUQ8S2d548SvKDLzBGTwqRU2S4OPmBArGLdvdDz8rts+GH1pSdlFBaaFVPDaQIDAQABAoIBAEUsIv7MfX/o7TifJnabGfkSOEM21ej8wOAGk3EwhO3LB6TXs9etuqsUH+HmCI/ATjOxTOpm22nG+y/dbCDU9MyeefzwnwYK8YlOIrfimGTpg1nNxQjby/hqWj5wqPf7xjWuDdn7RgGHNVcBcxirUwuw1g1KfJ/m8y+z6lKDIAWMuPegPFgQy0UoJmE5gjtdNYuRrPKESfjdgYhbmzl75k2zqm35Ngwgvp6YYq1jeGpDb4lDBDvn9KdpScC1y9w++7k4n1AyMZXsfgn3oSiFp9G6rZNraykOPYkQu309DVBqYtW0DHSU/xDYh1MTwJEwhcISYu12s2PIDGv/prgMRwUCgYEA7SdaqLT0B/btPkO84gnRx40rgsSM8gPewiVHerc95/tR6tCdMg1eNGJEK+biZMR/oxLQ3Ajr14BE3O8Dxhcqx/5vdo5qrX2oytDkl87oObK5rL0kdlmg/SQdnCsG/GkGtZlXLdMmjibSglGn23E69bsS0+IHspZnT2KHb1v1OZcCgYEA1ejfdHxmyOe+ke9QYn0umLLI/u6vDm6qkzEJrmzkpjrQrwftYRBeSr7CRJdRWtQ6dKA6kGZEfumFMg0ptFtwDGuLnzXek8UC3gKXjDnHyTugTXLprgB3A1AUYy0jvxmMTY8/AZLmDnqXma1WFnyxIUrTbzQq6uJPD4b33cWciv8CgYEAumnT1ocex1/uzqG6SEeFsYEjMZBEZjxqjlt1W13MeJxRoO1Ikz50zWJsycGcNa9L0SiKKluM3wGBn9T1N3GgfEJg5WU/L4517q7S8Q1/91KopsKqdakwZatM5yPfQutfjcGyCGBQjy6vDCcZdeIEgYICY7DpchTNslX1tbAoC5MCgYA9f9hOyz1Z4Zbeqik4R7lP2YcEFGdsBNExxFV+Onx6dkptKCBNWcFiR/necorHTGEKCs8LmPt0aXsL6tDks61BROI9geVeIrQyVBhyDmKsLmJmIfWhOyz8XNefs+ilFplJ6zc4Ip3V59USL82iZXMfmT20qRD1ut70Hd/BeQEKzQKBgQCoiTGlal7FaOHZmjvPOc6lzvOC2RIZL3yT5U1r9XsMFC2pPU/YinTc0cEpMmbeqLKuINjKOYyVp8HZEdpB6atU/WYDT2INe7VaphWpHkd5F56plzo0hlTDr1eFlHBsj23MVFR/UvpL0PeGzfnBd7ga2s0ymWDDnIhMJKzwu5GvDw==\n-----END RSA PRIVATE KEY-----" }, "cssl.acme_labs": { "certificates": [ { "certificate": "www.acmelabs.com" } ], "ciphers": "RSA", "class": "TLS_Server", "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": false }, "full_uri_decode": { "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICMgZGVjb2RlIG9yaWdpbmFsIFVSSS4KICBzZXQgdG1wVXJpIFtIVFRQOjp1cmldCiAgc2V0IHVyaSBbVVJJOjpkZWNvZGUgJHRtcFVyaV0KICAjIHJlcGVhdCBkZWNvZGluZyB1bnRpbCB0aGUgZGVjb2RlZCB2ZXJzaW9uIGVxdWFscyB0aGUgcHJldmlvdXMgdmFsdWUuCiAgd2hpbGUgeyAkdXJpIG5lICR0bXBVcmkgfSB7CiAgICBzZXQgdG1wVXJpICR1cmkKICAgIHNldCB1cmkgW1VSSTo6ZGVjb2RlICR0bXBVcmldCiAgfQogIEhUVFA6OnVyaSAkdXJpCiAgbG9nIGxvY2FsMC4gIk9yaWdpbmFsIFVSSTogW0hUVFA6OnVyaV0iCiAgbG9nIGxvY2FsMC4gIkZ1bGx5IGRlY29kZWQgVVJJOiAkdXJpIgp9" } } } } } OK, we have our declarations handy, now we can move on to working this the API endpoints! CRUD operations We sure love our acronyms in tech, don't we? CRUD stands for create, read, update, and delete. These are the most common operations for interacting with an API. (If you've used the iControl REST interface before on classic BIG-IP, you know that we need to perform additional operations like running commands (load, save, run, etc), so that needed to be folded in somehow to the CRUD model. We'll address those use cases in future articles.) Before we can use the API endpoints, however, we need to be authenticated to the Central Manager. This requires a login request that returns a bearer token to be used in subsequent requests. I wrote a short bash script to get the token which I set to a local variable in my shell. First, the script: #!/bin/zsh token=$(curl -ks --location 'https://172.16.31.105/api/login' \ --header 'Content-Type: application/json' \ --data '{ "username": "admin", "password": "notsofastmyfriend" }' | jq -r '.access_token') echo $token Next, setting the token variable for use in future commands: jrahm@mymac as3testing % token=$(./gt.sh) jrahm@mymac as3testing % echo $token D1RrEpn1RCHpm5FrGCIiXrwu3coSO8vWGT8e8kHLd2QbeUUiGAgw6pFb1B2l2bHeG7KsrqiipfuNGbx/DaCyUDQ0niaDiQizHIj6w7xOIWLNd5e/Bz2emGskM959E7CnMRTV36qPpu0SLDJsdvThZf6wLvm9oe5cX25Uqzf2/6Y+eNxDLs2WjsA4IFFRO2QWkjrq807kxJIoIX8BvICSxyjlx7PEQkWBAdUV7z6zayX03FtA3lqR66dzzMtIr9L7na+T7/i5cqSETGYQYt1z4a996oA/jMcAEy5J6PsuinCdN3ZZNt5Bfi4ck/5/bA3RJEZR8niU5u77DGasckdcUlRjl0/8UOgmEq19BRopAGFCXvRyiX/g6CVR6NDNG5dlmVjVcJ2+IzYJ8utGfr7raKMIgDIEn/G1AVqy0kj+x2ANdHpo0PQG678JoXChHObiDwjcOMrUiW2cC/YMLp36lcBEgp0uySokSwwYBTJjLJezFE74I+x154yDIWYD0+I8xbIqAHA4a3IxMljR14wowIJp84SxfeuJcrcUAZESzw== Now that I don't have to worry about re-upping on my token while working with curl at the command line, let's work through each of these CRUD operations in order. Application service create operation The create operation is accomplished with an HTTP POST method. As we are creating an object, we need to send some data along with that. That data in our case is the AS3 declaration. I put each declaration in a file Compatibility API It's a single request to deploy the workload with the compatibility interface to the /api/v1/spaces/default/appsvcs/declare endpoint with a target_address of the instances as a query parameter. Interestingly, the successful declaration is returned to you in its entirety in the response. DNS App jrahm@mymac as3testing % curl -sk \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d "@dns-app.json" \ --location 'https://172.16.2.105/api/v1/spaces/default/appsvcs/declare?target_address=172.16.2.161' | jq . { "declaration": { "class": "ADC", "id": "urn:uuid:3a71dceb-f56c-4dc1-901a-2feae0244c46", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "schemaVersion": "3.37.0", "tenant1": { "class": "Tenant", "dnsapp1": { "class": "Application", "pool.ns-cluster-1": { "class": "Pool", "members": [ { "addressDiscovery": "static", "serverAddresses": [ "10.10.100.101", "10.10.100.102", "10.10.100.103", "10.10.100.104" ], "servicePort": 53, "shareNodes": true } ], "monitors": [ "icmp" ] }, "template": "shared", "vip.ns-cluster-1": { "class": "Service_UDP", "pool": "pool.ns-cluster-1", "snat": "auto", "translateServerAddress": true, "virtualAddresses": [ "10.100.100.100" ], "virtualPort": 53 } } } }, "results": [ { "code": 200, "host": "172.16.2.161", "message": "success", "runTime": 1948, "tenant": "tenant1" } ] } HTTPS App jrahm@mymac as3testing % curl -sk \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d "@https-app.json" \ --location 'https://172.16.2.105/api/v1/spaces/default/appsvcs/declare?target_address=172.16.2.161' | jq . { "declaration": { "class": "ADC", "id": "urn:uuid:bd9c9728-8c20-4c4d-a625-68450e35e133", "label": "Converted Declaration", "remark": "Generated by Automation Config Converter", "schemaVersion": "3.37.0", "tenant2": { "class": "Tenant", "httpsapp1": { "class": "Application", "cssl.acme_labs": { "certificates": [ { "certificate": "www.acmelabs.com" } ], "ciphers": "RSA", "class": "TLS_Server", "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": false }, "full_uri_decode": { "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICMgZGVjb2RlIG9yaWdpbmFsIFVSSS4KICBzZXQgdG1wVXJpIFtIVFRQOjp1cmldCiAgc2V0IHVyaSBbVVJJOjpkZWNvZGUgJHRtcFVyaV0KICAjIHJlcGVhdCBkZWNvZGluZyB1bnRpbCB0aGUgZGVjb2RlZCB2ZXJzaW9uIGVxdWFscyB0aGUgcHJldmlvdXMgdmFsdWUuCiAgd2hpbGUgeyAkdXJpIG5lICR0bXBVcmkgfSB7CiAgICBzZXQgdG1wVXJpICR1cmkKICAgIHNldCB1cmkgW1VSSTo6ZGVjb2RlICR0bXBVcmldCiAgfQogIEhUVFA6OnVyaSAkdXJpCiAgbG9nIGxvY2FsMC4gIk9yaWdpbmFsIFVSSTogW0hUVFA6OnVyaV0iCiAgbG9nIGxvY2FsMC4gIkZ1bGx5IGRlY29kZWQgVVJJOiAkdXJpIgp9" } }, "pool.acme_labs": { "class": "Pool", "loadBalancingMode": "least-connections-member", "members": [ { "addressDiscovery": "static", "serverAddresses": [ "172.16.102.5" ], "servicePort": 80, "shareNodes": true } ], "monitors": [ "http" ] }, "template": "shared", "vip.acme_labs": { "class": "Service_HTTPS", "iRules": [ "full_uri_decode" ], "pool": "pool.acme_labs", "redirect80": false, "serverTLS": "cssl.acme_labs", "snat": "auto", "translateServerAddress": true, "virtualAddresses": [ "172.16.101.133" ], "virtualPort": 443 }, "www.acmelabs.com": { "certificate": "-----BEGIN CERTIFICATE-----\nMIIC3DCCAcSgAwIBAgIGAZAW7PncMA0GCSqGSIb3DQEBDQUAMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDAeFw0yNDA2MTQxMzI1NDdaFw0zNDA2MTIxMzI1NDdaMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMYpeRm4f1mPgW7STMM4gZXZ5p02nCWshNwVkaOLpRJAOdR2ZpuhLW4tWpAssvmTRlS0cFjZKA6ecVg4Q7+wvw7dIG8gVAviOqmHb6sDaomBTn3+ISFYW0Uxb1GNvZqlktJQI7hCsaS5Kf/f4pImVa8jQffWTdgLwxCm+0suaXy1XykVOCdOs1lsCOHjMoVREWxLIAtzMpqdO+8IRhSJgPJPf3GnY861T0LDjuT5rgwY1qK/H2NuEcPWOWVtqTN9aQAz9cKxDbJq48U8adzrl6G8uUYlEPEtneePErygy8wRk8KkVNkuDj5gQKxi3b3Q8/K7bPhh9aUnZRQWmhVTw2kCAwEAATANBgkqhkiG9w0BAQ0FAAOCAQEAOh3doWxnjb5j5XojnEtYUWJG6yw9a3xZhEiq7myWz7apmy5eAe0QAL9kFAuiBwgjqwzPCXzMDp21FdLC+o9Znx5A8kXE2W2G+h36kc21f3v0jumRdkU1zZ9py9iKHAOUSAYsALNWH4mosFFbodpqcFZL7Fqmh/AoIcqY3GqSWOZ6geYbMIOwTZFnsuE1LTjJrnypz1ZyglGoftzU9j501aq3eJ3YUyRIZ28/ARJxn4sUfdvjvs31EdFEOOC6hwN2U7JXdWWK/fATTenglSkUqChJRW6kRL7uFf6FCCZjXyGINJnOYVz+8gxDWA557+ogYfEquQVML5gvMK9Ff67W6A==\n-----END CERTIFICATE-----", "class": "Certificate", "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAxil5Gbh/WY+BbtJMwziBldnmnTacJayE3BWRo4ulEkA51HZmm6Etbi1akCyy+ZNGVLRwWNkoDp5xWDhDv7C/Dt0gbyBUC+I6qYdvqwNqiYFOff4hIVhbRTFvUY29mqWS0lAjuEKxpLkp/9/ikiZVryNB99ZN2AvDEKb7Sy5pfLVfKRU4J06zWWwI4eMyhVERbEsgC3Mymp077whGFImA8k9/cadjzrVPQsOO5PmuDBjWor8fY24Rw9Y5ZW2pM31pADP1wrENsmrjxTxp3OuXoby5RiUQ8S2d548SvKDLzBGTwqRU2S4OPmBArGLdvdDz8rts+GH1pSdlFBaaFVPDaQIDAQABAoIBAEUsIv7MfX/o7TifJnabGfkSOEM21ej8wOAGk3EwhO3LB6TXs9etuqsUH+HmCI/ATjOxTOpm22nG+y/dbCDU9MyeefzwnwYK8YlOIrfimGTpg1nNxQjby/hqWj5wqPf7xjWuDdn7RgGHNVcBcxirUwuw1g1KfJ/m8y+z6lKDIAWMuPegPFgQy0UoJmE5gjtdNYuRrPKESfjdgYhbmzl75k2zqm35Ngwgvp6YYq1jeGpDb4lDBDvn9KdpScC1y9w++7k4n1AyMZXsfgn3oSiFp9G6rZNraykOPYkQu309DVBqYtW0DHSU/xDYh1MTwJEwhcISYu12s2PIDGv/prgMRwUCgYEA7SdaqLT0B/btPkO84gnRx40rgsSM8gPewiVHerc95/tR6tCdMg1eNGJEK+biZMR/oxLQ3Ajr14BE3O8Dxhcqx/5vdo5qrX2oytDkl87oObK5rL0kdlmg/SQdnCsG/GkGtZlXLdMmjibSglGn23E69bsS0+IHspZnT2KHb1v1OZcCgYEA1ejfdHxmyOe+ke9QYn0umLLI/u6vDm6qkzEJrmzkpjrQrwftYRBeSr7CRJdRWtQ6dKA6kGZEfumFMg0ptFtwDGuLnzXek8UC3gKXjDnHyTugTXLprgB3A1AUYy0jvxmMTY8/AZLmDnqXma1WFnyxIUrTbzQq6uJPD4b33cWciv8CgYEAumnT1ocex1/uzqG6SEeFsYEjMZBEZjxqjlt1W13MeJxRoO1Ikz50zWJsycGcNa9L0SiKKluM3wGBn9T1N3GgfEJg5WU/L4517q7S8Q1/91KopsKqdakwZatM5yPfQutfjcGyCGBQjy6vDCcZdeIEgYICY7DpchTNslX1tbAoC5MCgYA9f9hOyz1Z4Zbeqik4R7lP2YcEFGdsBNExxFV+Onx6dkptKCBNWcFiR/necorHTGEKCs8LmPt0aXsL6tDks61BROI9geVeIrQyVBhyDmKsLmJmIfWhOyz8XNefs+ilFplJ6zc4Ip3V59USL82iZXMfmT20qRD1ut70Hd/BeQEKzQKBgQCoiTGlal7FaOHZmjvPOc6lzvOC2RIZL3yT5U1r9XsMFC2pPU/YinTc0cEpMmbeqLKuINjKOYyVp8HZEdpB6atU/WYDT2INe7VaphWpHkd5F56plzo0hlTDr1eFlHBsj23MVFR/UvpL0PeGzfnBd7ga2s0ymWDDnIhMJKzwu5GvDw==\n-----END RSA PRIVATE KEY-----" } } } }, "results": [ { "code": 200, "host": "172.16.2.161", "message": "success", "runTime": 1950, "tenant": "tenant2" } ] } Documents API With this approach, you send the document first with the /api/v1/spaces/default/appsvcs/documents endpoint and then deploy with the /api/v1/spaces/default/appsvcs/documents/<id>/deployments endpoint. The document and deployment each have their own object ID, and then the deployment also has a task ID that can be referenced in the logs. DNS App jrahm@mymac as3testing % curl -skX POST \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d "@dns-app.json" \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents | jq . { "Message": "Application service created successfully", "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3" } }, "id": "d5d0a360-75ec-434c-9802-62083a26c4d3" } jrahm@mymac as3testing % curl -skX POST \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d '{"target": "172.16.2.161"}' \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/deployments | jq . { "Message": "Deployment task created successfully", "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/deployments" } }, "id": "ed48899b-fcb0-4a60-b8f2-2c0e012aa28d", "task_id": "771beda9-5ca4-4049-bebc-97b9d52da524" } HTTPS App jrahm@mymac as3testing % curl -skX POST \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d "@https-app.json" \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents | jq . { "Message": "Application service created successfully", "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab" } }, "id": "3102ce15-e3d4-498f-a466-60f4bf02c2ab" } jrahm@mymac as3testing % curl -skX POST \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d '{"target": "172.16.2.161"}' \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab/deployments | jq . { "Message": "Deployment task created successfully", "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab/deployments" } }, "id": "400e2b06-b451-4035-a26b-beaf90b283a5", "task_id": "f529800a-f515-4bec-9cfe-1f3214dec229" } Central Manager view of API-deployed apps This is the result in Central Manager after deploying the two applications via the two different methodologies. Notice the different naming scheme applied to each approach. Application service read operation The read operation is accomplished with an HTTP GET method. No payload is necessary on the request. Compatibility API Note here that both the DNS and HTTP apps will be returned, and for that matter, both could have been deployed together as well! Also note that this is for apps on the targeted instance only, however. The AS3 deployments follow the curl command options. jrahm@mymac as3testing % curl -sk \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ "https://172.16.2.105/api/v1/spaces/default/appsvcs/declare?target_address=172.16.2.161" | jq . { "class": "ADC", "controls": null, "schemaVersion": "3.0.0", "target": { "address": "172.16.2.161" }, "tenant1": { "class": "Tenant", "dnsapp1": { "class": "Application", "pool.ns-cluster-1": { "class": "Pool", "members": [ { "addressDiscovery": "static", "serverAddresses": [ "10.10.100.101", "10.10.100.102", "10.10.100.103", "10.10.100.104" ], "servicePort": 53, "shareNodes": true } ], "monitors": [ "icmp" ] }, "template": "shared", "vip.ns-cluster-1": { "class": "Service_UDP", "pool": "pool.ns-cluster-1", "snat": "auto", "translateServerAddress": true, "virtualAddresses": [ "10.100.100.101" ], "virtualPort": 53 } } }, "tenant2": { "class": "Tenant", "httpsapp1": { "class": "Application", "cssl.acme_labs": { "certificates": [ { "certificate": "www.acmelabs.com" } ], "ciphers": "RSA", "class": "TLS_Server", "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": false }, "full_uri_decode": { "class": "iRule", "iRule": { "base64": "d2hlbiBIVFRQX1JFUVVFU1QgewogICMgZGVjb2RlIG9yaWdpbmFsIFVSSS4KICBzZXQgdG1wVXJpIFtIVFRQOjp1cmldCiAgc2V0IHVyaSBbVVJJOjpkZWNvZGUgJHRtcFVyaV0KICAjIHJlcGVhdCBkZWNvZGluZyB1bnRpbCB0aGUgZGVjb2RlZCB2ZXJzaW9uIGVxdWFscyB0aGUgcHJldmlvdXMgdmFsdWUuCiAgd2hpbGUgeyAkdXJpIG5lICR0bXBVcmkgfSB7CiAgICBzZXQgdG1wVXJpICR1cmkKICAgIHNldCB1cmkgW1VSSTo6ZGVjb2RlICR0bXBVcmldCiAgfQogIEhUVFA6OnVyaSAkdXJpCiAgbG9nIGxvY2FsMC4gIk9yaWdpbmFsIFVSSTogW0hUVFA6OnVyaV0iCiAgbG9nIGxvY2FsMC4gIkZ1bGx5IGRlY29kZWQgVVJJOiAkdXJpIgp9" } }, "pool.acme_labs": { "class": "Pool", "loadBalancingMode": "least-connections-member", "members": [ { "addressDiscovery": "static", "serverAddresses": [ "172.16.102.5" ], "servicePort": 80, "shareNodes": true } ], "monitors": [ "http" ] }, "template": "shared", "vip.acme_labs": { "class": "Service_HTTPS", "iRules": [ "full_uri_decode" ], "pool": "pool.acme_labs", "redirect80": false, "serverTLS": "cssl.acme_labs", "snat": "auto", "translateServerAddress": true, "virtualAddresses": [ "172.16.101.133" ], "virtualPort": 443 }, "www.acmelabs.com": { "certificate": "-----BEGIN CERTIFICATE-----\nMIIC3DCCAcSgAwIBAgIGAZAW7PncMA0GCSqGSIb3DQEBDQUAMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDAeFw0yNDA2MTQxMzI1NDdaFw0zNDA2MTIxMzI1NDdaMC8xCzAJBgNVBAYTAlVTMSAwHgYDVQQDExdteXNlbGZzaWduZWQudGVzdC5sb2NhbDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAMYpeRm4f1mPgW7STMM4gZXZ5p02nCWshNwVkaOLpRJAOdR2ZpuhLW4tWpAssvmTRlS0cFjZKA6ecVg4Q7+wvw7dIG8gVAviOqmHb6sDaomBTn3+ISFYW0Uxb1GNvZqlktJQI7hCsaS5Kf/f4pImVa8jQffWTdgLwxCm+0suaXy1XykVOCdOs1lsCOHjMoVREWxLIAtzMpqdO+8IRhSJgPJPf3GnY861T0LDjuT5rgwY1qK/H2NuEcPWOWVtqTN9aQAz9cKxDbJq48U8adzrl6G8uUYlEPEtneePErygy8wRk8KkVNkuDj5gQKxi3b3Q8/K7bPhh9aUnZRQWmhVTw2kCAwEAATANBgkqhkiG9w0BAQ0FAAOCAQEAOh3doWxnjb5j5XojnEtYUWJG6yw9a3xZhEiq7myWz7apmy5eAe0QAL9kFAuiBwgjqwzPCXzMDp21FdLC+o9Znx5A8kXE2W2G+h36kc21f3v0jumRdkU1zZ9py9iKHAOUSAYsALNWH4mosFFbodpqcFZL7Fqmh/AoIcqY3GqSWOZ6geYbMIOwTZFnsuE1LTjJrnypz1ZyglGoftzU9j501aq3eJ3YUyRIZ28/ARJxn4sUfdvjvs31EdFEOOC6hwN2U7JXdWWK/fATTenglSkUqChJRW6kRL7uFf6FCCZjXyGINJnOYVz+8gxDWA557+ogYfEquQVML5gvMK9Ff67W6A==\n-----END CERTIFICATE-----", "class": "Certificate", "privateKey": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCAQEAxil5Gbh/WY+BbtJMwziBldnmnTacJayE3BWRo4ulEkA51HZmm6Etbi1akCyy+ZNGVLRwWNkoDp5xWDhDv7C/Dt0gbyBUC+I6qYdvqwNqiYFOff4hIVhbRTFvUY29mqWS0lAjuEKxpLkp/9/ikiZVryNB99ZN2AvDEKb7Sy5pfLVfKRU4J06zWWwI4eMyhVERbEsgC3Mymp077whGFImA8k9/cadjzrVPQsOO5PmuDBjWor8fY24Rw9Y5ZW2pM31pADP1wrENsmrjxTxp3OuXoby5RiUQ8S2d548SvKDLzBGTwqRU2S4OPmBArGLdvdDz8rts+GH1pSdlFBaaFVPDaQIDAQABAoIBAEUsIv7MfX/o7TifJnabGfkSOEM21ej8wOAGk3EwhO3LB6TXs9etuqsUH+HmCI/ATjOxTOpm22nG+y/dbCDU9MyeefzwnwYK8YlOIrfimGTpg1nNxQjby/hqWj5wqPf7xjWuDdn7RgGHNVcBcxirUwuw1g1KfJ/m8y+z6lKDIAWMuPegPFgQy0UoJmE5gjtdNYuRrPKESfjdgYhbmzl75k2zqm35Ngwgvp6YYq1jeGpDb4lDBDvn9KdpScC1y9w++7k4n1AyMZXsfgn3oSiFp9G6rZNraykOPYkQu309DVBqYtW0DHSU/xDYh1MTwJEwhcISYu12s2PIDGv/prgMRwUCgYEA7SdaqLT0B/btPkO84gnRx40rgsSM8gPewiVHerc95/tR6tCdMg1eNGJEK+biZMR/oxLQ3Ajr14BE3O8Dxhcqx/5vdo5qrX2oytDkl87oObK5rL0kdlmg/SQdnCsG/GkGtZlXLdMmjibSglGn23E69bsS0+IHspZnT2KHb1v1OZcCgYEA1ejfdHxmyOe+ke9QYn0umLLI/u6vDm6qkzEJrmzkpjrQrwftYRBeSr7CRJdRWtQ6dKA6kGZEfumFMg0ptFtwDGuLnzXek8UC3gKXjDnHyTugTXLprgB3A1AUYy0jvxmMTY8/AZLmDnqXma1WFnyxIUrTbzQq6uJPD4b33cWciv8CgYEAumnT1ocex1/uzqG6SEeFsYEjMZBEZjxqjlt1W13MeJxRoO1Ikz50zWJsycGcNa9L0SiKKluM3wGBn9T1N3GgfEJg5WU/L4517q7S8Q1/91KopsKqdakwZatM5yPfQutfjcGyCGBQjy6vDCcZdeIEgYICY7DpchTNslX1tbAoC5MCgYA9f9hOyz1Z4Zbeqik4R7lP2YcEFGdsBNExxFV+Onx6dkptKCBNWcFiR/necorHTGEKCs8LmPt0aXsL6tDks61BROI9geVeIrQyVBhyDmKsLmJmIfWhOyz8XNefs+ilFplJ6zc4Ip3V59USL82iZXMfmT20qRD1ut70Hd/BeQEKzQKBgQCoiTGlal7FaOHZmjvPOc6lzvOC2RIZL3yT5U1r9XsMFC2pPU/YinTc0cEpMmbeqLKuINjKOYyVp8HZEdpB6atU/WYDT2INe7VaphWpHkd5F56plzo0hlTDr1eFlHBsj23MVFR/UvpL0PeGzfnBd7ga2s0ymWDDnIhMJKzwu5GvDw==\n-----END RSA PRIVATE KEY-----" } } } } Documents API With this interface, Central Manager lists out all the documents, including the compatibility interface applications. jrahm@mymac as3testing % curl -sk \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents | jq ._embedded.appsvcs [ { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab" } }, "created": "2024-06-17T17:38:08.186126Z", "deployments": [ { "id": "400e2b06-b451-4035-a26b-beaf90b283a5", "instance_id": "a4148c93-5306-4605-b8bb-92d6b1f78c26", "target": { "instance_ip": "172.16.2.161" }, "last_successful_deploy_time": "2024-06-17T17:38:42.404675Z", "modified": "2024-06-17T17:38:42.404675Z", "last_record": { "id": "64894415-38d0-49f9-989d-8f00c88196b3", "task_id": "f529800a-f515-4bec-9cfe-1f3214dec229", "start_time": "2024-06-17T17:38:41.103539Z", "status": "completed" } } ], "deployments_count": { "total": 1, "completed": 1 }, "id": "3102ce15-e3d4-498f-a466-60f4bf02c2ab", "name": "httpsapp1", "tenant_name": "tenant2", "type": "AS3" }, { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/7938a0a2-b5d4-4687-99f8-e73d9e6b3d51" } }, "created": "2024-06-17T17:52:41.397543Z", "deployments": [ { "id": "0c50d882-f8d1-4833-af31-2b71e465f2f5", "instance_id": "a4148c93-5306-4605-b8bb-92d6b1f78c26", "target": { "instance_ip": "172.16.2.161" }, "last_successful_deploy_time": "2024-06-17T17:54:51.531445Z", "modified": "2024-06-17T17:54:51.531445Z", "last_record": { "id": "a8f786a5-f1c6-4f99-83bb-59cc024e1c34", "task_id": "ee1a3afa-c9d4-4e29-9271-632bbb93b6e7", "start_time": "2024-06-17T17:54:50.167979Z", "status": "completed" } } ], "deployments_count": { "total": 1, "completed": 1 }, "id": "7938a0a2-b5d4-4687-99f8-e73d9e6b3d51", "modified": "2024-06-17T17:54:50.164813Z", "name": "tenant1.dnsapp1.NzKPI4xZ", "tenant_name": "default", "type": "AS3" }, { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/87ec6d3a-063d-4660-b32a-08cf183a21a8" } }, "created": "2024-06-17T17:50:02.621622Z", "deployments": [ { "id": "5da24b69-491e-45a1-b8eb-18395c4b2b12", "instance_id": "a4148c93-5306-4605-b8bb-92d6b1f78c26", "target": { "instance_ip": "172.16.2.161" }, "last_successful_deploy_time": "2024-06-17T17:50:03.929715Z", "modified": "2024-06-17T17:50:03.929715Z", "last_record": { "id": "1f3bc580-da07-4c26-b4d2-7e8bcb632869", "task_id": "dc8fbdc8-4dd0-4aeb-9e7d-cf3038d42c07", "start_time": "2024-06-17T17:50:02.640417Z", "status": "completed" } } ], "deployments_count": { "total": 1, "completed": 1 }, "id": "87ec6d3a-063d-4660-b32a-08cf183a21a8", "name": "tenant2.httpsapp1.NzKPI4xZ", "tenant_name": "default", "type": "AS3" }, { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3" } }, "created": "2024-06-17T17:56:04.957896Z", "deployments": [ { "id": "ed48899b-fcb0-4a60-b8f2-2c0e012aa28d", "instance_id": "a4148c93-5306-4605-b8bb-92d6b1f78c26", "target": { "instance_ip": "172.16.2.161" }, "last_successful_deploy_time": "2024-06-17T17:56:34.410606Z", "modified": "2024-06-17T17:56:34.410606Z", "last_record": { "id": "7178d940-5ae7-4c18-bca6-6f7d14604d5e", "task_id": "771beda9-5ca4-4049-bebc-97b9d52da524", "start_time": "2024-06-17T17:56:33.123687Z", "status": "completed" } } ], "deployments_count": { "total": 1, "completed": 1 }, "id": "d5d0a360-75ec-434c-9802-62083a26c4d3", "name": "dnsapp1", "tenant_name": "tenant1", "type": "AS3" } ] Application service update operation For the update operation, this could be an HTTP PUT or PATCH method, depending on what the endpoints support. PUT is supposed to be a total replacement and PATCH a partial replacement, but I've found the implementations of many APIs to not follow this pattern. These methods require a payload with the request. In this section forward, we'll focus more on the mechanics of the API rather than the specifics on the application services, so I might work with one or the other unless both need attention. Compatibility API This is where I throw a curveball at you! As the compatibility interface is intended to match BIG-IP classic AS3 behavior so it is in fact, uh, compatible, the operation for an update is actually still a POST as if you're creating the application service for the first time, so there's no need to do anything new here. Make the change to your declaration and POST as shown in the create section and you're good to go. Documents API To modify the AS3 application service, the API reference states that the PUT method should be used, and the declaration should be complete. So I changed the virtual server IP address in the declaration and sent a PUT request to the appropriate document ID and it was successfully deployed. jrahm@mymac as3testing % curl -skX PUT \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d "@dns-app.json" \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3 | jq . { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3" } }, "deployments": [ { "Message": "Update deployment task created", "id": "ed48899b-fcb0-4a60-b8f2-2c0e012aa28d", "task_id": "b07fa2de-7d73-4c7e-988a-1383cc45e441" } ], "id": "d5d0a360-75ec-434c-9802-62083a26c4d3", "message": "Application service updated successfully" } Application service delete operation An HTTP DELETE method performs the delete operation. Typically you just need the object ID in the request URL to remove the desired object. This is the fun part, at least in the lab environment. BLOW STUFF UP! Just kidding, but not really. I, like the Joker before me, like to make things go bye bye. Maybe if the Joker could have been a force for good he'd be a great chaos engineer. Compatibility API This is where I put up the RED FLAG and caution you to know what you're doing here. If you send a DELETE to the compatibility interface with an empty payload you can blow away ALL the AS3 configuration on that instance. So don't do that... Instead, make sure you include the tenant name in the URI as shown below. jrahm@mymac as3testing % curl -skX DELETE \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ --location 'https://172.16.2.105/api/v1/spaces/default/appsvcs/declare/tenant1?target_address=172.16.2.161' { "declaration":{}, "results":[ { "code":200, "host":"172.16.2.161", "message":"success", "runTime":1331, "tenant":"tenant1" } ] } Documents API You have two options here. You can delete the deployment only (you'll need to provide the document ID and the deployment ID) and then choose whether to the leave the draft or delete it (I show the document delete as well): jrahm@mymac as3testing % curl -skX DELETE \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d '{"target": "172.16.2.161"}' \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/deployments/ed48899b-fcb0-4a60-b8f2-2c0e012aa28d | jq . { "Message": "Delete Deployment task created successfully", "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/deployments/ed48899b-fcb0-4a60-b8f2-2c0e012aa28d" } }, "id": "ed48899b-fcb0-4a60-b8f2-2c0e012aa28d", "task_id": "9c5a8fe0-d8b9-4b41-a47f-3283586c88f1" } jrahm@mymac as3testing % curl -skX DELETE \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d '{"target": "172.16.2.161"}' \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/ | jq . { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/d5d0a360-75ec-434c-9802-62083a26c4d3/" } }, "id": "d5d0a360-75ec-434c-9802-62083a26c4d3", "message": "The application has been deleted successfully" } Or you can delete the document outright in one step which will clean up the deployment as well: jrahm@mymac as3testing % curl -skX DELETE \ -H "Authorization: Bearer $token" \ -H "Content-Type: application/json" \ -d '{"target": "172.16.2.161"}' \ https://172.16.2.105/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab/ | jq . { "_links": { "self": { "href": "/api/v1/spaces/default/appsvcs/documents/3102ce15-e3d4-498f-a466-60f4bf02c2ab/" } }, "deployments": [ { "Message": "Delete Deployment task created successfully", "id": "/declare/3102ce15-e3d4-498f-a466-60f4bf02c2ab/deployments/400e2b06-b451-4035-a26b-beaf90b283a5", "task_id": "73001fa0-2690-4906-96d6-52c2bb162bb0" } ], "id": "3102ce15-e3d4-498f-a466-60f4bf02c2ab", "message": "The application delete has been submitted successfully" } One more AS3 schema insight This article focused on the API endpoints and to make things simpler I used a declaration that works with both approaches. That said, if you are starting out with BIG-IP Next, you don't need the ADC or Tenant classes in your declaration, you can instead use a named document and start at the application class. Check out this diff in VSCode for the DNS app used in this article. Next up... I've been configuration-focused in the first couple of articles in the automation series. In the next article, I'll walk through some of the BIG-IP Next Postman collection, looking at system as well as configuration things. The visual experience in Postman might be a little easier on the eyes for those getting started than a bunch of curl commands. Stay tuned! Resources BIG-IP Next AS3 Schema BIG-IP Next API Reference Manage Application Services on Central Manager with AS3Embracing AS3: Foundations
(updated to remove the event-nature of this post) Last fall, a host of teams took to the road to support the launch of BIG-IP Next in the form of F5 Academy roadshows, where we shared the BIG-IP story: where we started, where we are, and where we're going with it; complete with hands-on LTM and WAF labs with the attendees. For this spring's roadshows, we added SSLO and Access labs. Across the fall and spring legs, I attended in person in Kansas City (twice!), St. Louis, Cincinnati, Columbus, Omaha, and (soon) Chicago and talked with customers at all stages of the automation journey. Some haven't automated much of anything. Some have been using a variety of on-/off-box scripts, and some are all-in, baby! That said, when I ask about AS3 as a tool in their tool belt, not that many have adopted or even investigated it yet. For classic BIG-IP AS3 is not a requirement, but in BIG-IP Next, AS3 is a critical component as it's THE underlying configuration language for all applications. Because of this, I did a five-part live stream series in December to get you started with AS3. Details below. Beyond Imperatives—What the heck is AS3? In this first episode, I covered the history of automation on BIG-IP, the differences between imperative and declarative models, and the basics of AS3 from data structure and systems architecture perspectives. Top 10 Features to Know in the VSCode F5 Extension Special guest and friend of the show Ben Novak, author of the F5 Extension for VSCode, joined me to dig into the features most important to know and learn to use for understanding how to take stock BIG-IP configurations and turn applications into declarations. Migrating and Deploying Applications in VSCode In this episode, armed with the knowledge I learned from Ben in the last episode, I dug into the brass tacks of AS3! I reviewed diagnostics and migrated applications from standard configuration to AS3 declarations and deployed as well. For migrating active workloads, I discussed the steps necessary to reduce transition impact. Creating New Apps and Using Shared Objects I've always tried to alter existing things before creating new things with tech, and this is no different. Now that I had a few migrations under my belt, I attacked a net-new application and looked at shared objects and how and when to use them. Best Practices Finally, I closed this series with a look at several best practices when working with AS3. The conclusion to this series, though, is hopefully just the stepping off point for everyone new to AS3, and we can continue the conversation right here on DevCentral.BIG-IP Orchestrate in private Data Center using Terraform Cloud Agent
What is Terraform Cloud? Terraform Cloud offers organizations a unified workflow for provisioning their cloud, private data center, and SaaS infrastructure, ensuring continuous infrastructure management throughout its entire lifecycle. What is F5 BIG-IP? BIG-IP is a collection of hardware platforms and software solutions providing services focused on security, reliability, and performance. It helps in doing Application delivery server load balancing of applications securely and at scale. BIG-IP can be deployed in private or public clouds. BIG-IP in private Data Center When BIG-IP is in a private data center, it has a private IP, making it tricky to reach with tools like Terraform Cloud from the outside. However, if you're dealing with both private and public clouds using Terraform Cloud, you can use Terraform Cloud agents. These agents help control BIG-IP in private data centers, even when the IP isn't accessible externally. BIG-IP supports Application Services 3 (AS3) and FAST templates, presenting a highly synergistic relationship with Terraform. This synergy is particularly pronounced due to the availability of the BIG-IP Terraform provider, coupled with dedicated resources designed specifically for the deployment of AS3 and FAST templates. AS3 and FAST templates serve as powerful tools for configuring and managing BIG-IP application services. AS3 simplifies the process of defining, managing, and deploying application-related configurations, providing a declarative model for specifying how applications should be set up on BIG-IP devices. FAST, on the other hand, extends automation capabilities by incorporating telemetry functionalities, and enhancing monitoring and reporting capabilities. The integration with Terraform is pivotal in this context, as the BIG-IP Terraform provider facilitates the seamless incorporation of AS3 and FAST templates into infrastructure-as-code (IaC) workflows. The example Terraform configuration is at https://github.com/scshitole/privateDC How to orchestrate BIG-IP in a private Data Center? In this illustrative scenario, the BIG-IP system is operational within a private Data Center. A Virtual Machine has been configured to host the Terraform Cloud agent, encapsulated within a container. The Terraform configuration pertinent to our deployment resides in the GitHub repository: https://github.com/scshitole/privateDC. Let us now turn our attention to the Terraform Agent—an agile and lightweight component capable of executing within a container on a Virtual Machine. Its primary function is to establish and maintain a secure connection with Terraform Cloud, perpetually polling for instructions. Notably, this agent not only retrieves directives from Terraform Cloud but also acquires essential information regarding the Terraform workspace and the TF configuration. What distinguishes this agent is its seamless operation without necessitating alterations to existing firewall rules. Its communication with Terraform Cloud transpires over HTTPS and only demands appropriately configured DNS settings. Upon queuing a Terraform plan, the control plane initiates the dispatch of the configuration to the agent. Subsequently, the agent diligently retrieves the workspace and orchestrates the deployment of the configuration onto the BIG-IP infrastructure. What advantages does this solution offer? Automation Workflows with Terraform Cloud: By leveraging Terraform Cloud, we gain the capability to establish automated workflows for configuring BIG-IP. This not only streamlines the configuration process but also enhances efficiency through the power of automation. Additionally, Terraform Cloud enables the orchestration of BIG-IP Web Application Firewall (WAF) configurations in a Hybrid Cloud environment, providing a comprehensive solution for managing security across diverse infrastructures. Enhanced Security with Maintained Private IPs and Credentials: The solution ensures a robust security posture by maintaining the confidentiality of the infrastructure's private IP addresses and credentials. This practice prevents security sprawls and unauthorized access attempts, fortifying the integrity of the entire system. Seamless BIG-IP Configuration Migration: The flexibility of BIG-IP configuration migration is a notable advantage, allowing for a smooth transition between private and public cloud environments. This bidirectional migration capability ensures adaptability to evolving infrastructure needs, facilitating a seamless shift of BIG-IP configurations as organizational requirements dictate. Whether moving configurations from a private cloud to a public cloud or vice versa, this capability provides agility and scalability in infrastructure management. How to set up configuration on Terraform Cloud? o Once logged into Terraform Cloud, choose your organization from the available options. o Go to the Projects & Workspaces section and opt for the Version Control Workflow. o The BIG-IP Terraform configuration template resides in the GitHub repository; please choose the relevant repository. o Choose the correct GitHub repository; it should be visible here. o Now, provide a name for the workspace; feel free to select something relevant, but it must be unique. o Enter the variables here, including details such as the BIG-IP's IP address, username, and password. Ensure to choose the HCL option, and if needed, you can set it as invisible. o Then, go to the established workspace and click on "New run. o Go to the Agents section and select Create Agent Pool. o Enter a fitting name for the Agent Pool, as illustrated. You can opt for a unique name matching the workspace for easy identification, although it's not mandatory. o Provide a suitable description for the Agent Pool, explaining its specific purpose or activities. Then, proceed to click on "Generate Token"; you will require this token when running the agent. o Copy the newly generated token and follow the outlined steps to configure your agents. Run the provided docker command, including essential environment variables like TFC_AGENT_TOKEN and TFC_AGENT_NAME. If desired, you can also run the docker in the background using the appropriate docker command option. Key Take Away Terraform Cloud streamlines infrastructure provisioning and management across various environments for consistent lifecycle control. Terraform Cloud agents enable effective orchestration of BIG-IP configurations in private data centers, addressing challenges associated with private IPs. The seamless integration of BIG-IP, AS3, FAST templates, and Terraform supports efficient infrastructure-as-code workflows, especially beneficial in multi-cloud setups. Terraform Cloud facilitates automated workflows, simplifies BIG-IP configurations, and supports orchestrating Web Application Firewall (WAF) setups in Hybrid Cloud environments. Emphasizing security, the solution maintains the confidentiality of private IPs and credentials, preventing security sprawl and unauthorized access. The solution offers flexibility by allowing seamless BIG-IP configuration migration between private and public cloud environments, ensuring adaptability and scalability. For more details, please watch the accompanying video https://youtu.be/RgCqnDxpf3EGetting Started with BIG-IP Next: Migrating an Application Workload
So far in this article series, the focus has been completely on the operational readiness of BIG-IP Next as a system. In this article, I'll walk through migrating an application currently supported by my classic BIG-IP running TMOS version 15.1.x. The application is just a simple instance of an NGINX web server fronted on LTM with basic load balancing, TLS offloading, and a basic WAF policy. There are a lot of screenshots in this article, which might seem overwhelming. Doing your own walkthrough, however, will put your mind at ease; it actually moves pretty quickly in realtime. Existing Application Workload on TMOS We'll start with the GUI representation of the application workload. It is secured with TLS, which is offloaded at the BIG-IP with a clientssl profile and not re-encrypted to the server. There are custom TCP and HTTP profiles defined as well as the aforementioned custom clientssl profile. Snat automap is enabled, and a specific VLAN is configured to allow connections. On the security tab, an application security policy is enabled, and the log illegal requests log profile is enabled as well. Finally, under resources, the default pool is defined and a policy is in place to map requests to the applied security policy. On the CLI, that virtual server along with all the other referenced BIG-IP objects are defined in the tmsh version of that configuration. ltm virtual nginx-vip-tls { destination 172.16.101.50:https ip-protocol tcp mask 255.255.255.255 policies { asm_auto_l7_policy__nginx-vip-tls { } } pool nginx-pool profiles { ASM_testpol { } cssl.TestSuite { context clientside } customHTTP { } customTCP { } websecurity { } } security-log-profiles { "Log illegal requests" } source-address-translation { type automap } vlans { vlan.br1 } vlans-enabled } ltm policy asm_auto_l7_policy__nginx-vip-tls { controls { asm } last-modified 2024-03-20:13:25:13 requires { http } rules { default { actions { 1 { asm enable policy /Common/testpol } } ordinal 1 } } status legacy strategy first-match } ltm pool nginx-pool { members { 172.16.102.5:http { address 172.16.102.5 session monitor-enabled state up } } monitor http } security bot-defense asm-profile ASM_testpol { app-service none clientside-in-use disabled flags 0 inject-javascript disabled persistent-data-validity-period 0 send-brute-force-challenge disabled send-javascript-challenge disabled send-javascript-efoxy disabled send-javascript-fingerprint disabled } ltm profile client-ssl cssl.TestSuite { app-service none cert-key-chain { default { cert default.crt key default.key } } cipher-group cg_TLSv1.3 ciphers none defaults-from clientssl inherit-ca-certkeychain true inherit-certkeychain true options { dont-insert-empty-fragments } } ltm cipher group cg_TLSv1.3 { allow { cr_TLSv1.3 { } } } ltm cipher rule cr_TLSv1.3 { cipher TLSv1_3 dh-groups DEFAULT signature-algorithms DEFAULT } ltm profile http customHTTP { app-service none defaults-from http enforcement { known-methods { PATCH DELETE GET POST PUT } max-header-count 32 max-header-size 16384 rfc-compliance enabled } hsts { mode enabled } insert-xforwarded-for enabled proxy-type reverse } ltm profile tcp customTCP { app-service none congestion-control bbr defaults-from f5-tcp-progressive idle-timeout 600 ip-tos-to-client pass-through keep-alive-interval 2100 pkt-loss-ignore-burst 3 pkt-loss-ignore-rate 10 proxy-options enabled } ltm profile web-security websecurity { } You can see that I have some non-standard options in some of that configuration, such as specifying the congestion-control algorithm algorithm in the TCP profile, enabling HSTS in the HTTP profile, and setting cipher rules and groups for use in my SSL profile. Now that we have an idea of the workload we're going to migrate, let's create a UCS of the system for use in the migration. If you are already comfortable with this part on classic BIG-IP systems, you can skip down to the next section header. First, login to your classic BIG-IP and navigate to System->Archives and click Create. Give it a name and click Finished. I named mine next-migration. Click OK after the UCS has been generated and saved. In the archive list, click the name of the UCS you created. Click the Download button. Migrating the Workload in Central Manager Upload UCS and Analyze the Workloads Armed with your UCS, login to Central Manager and on the welcome screen, click Go to Application Workspace. If you have not added any applications yet, you'll see a screen like this with a Start Adding Apps button. If you already have something defined, you'll see a list of applications. Click the + Add Application button instead. On this screen, we'll bypass creating a new application service and select New Migration. Name your session as you'll be able to come back to it to migrate other applications later if your intent is to just migrate a single application for now (as is the case with this walkthrough.) I added a description but it is not necessary. Click Next. Here you'll select your UCS archive and group your application services by IP addresses OR by virtual server. I stuck with the recommended default. Click Next. Your UCS will now upload and then Central Manager will analyze and group the package. An enhanced version of the JOURNEYS tool available in the f5devcentral organization on GitHub is used here. Select Add Application. Application 5 is the one we are interested in analyzing and migrating for this walkthrough, so I selected that one. Notice in the status column the applications that have warnings, and that ours is one of them. Hovering over the triangle icon it indicates the app can be migrated, but without some of the functionality from our classic iteration of this workload. Next, click Analyze at the top right so we can see what can't be migrated. In the Configuration Analyzer screen, there are 3 files with areas of concern. First, that the websecurity profile is not supported. This is ok, the mechanisms to support attaching policies in Next are slightly different. Next from what was the bigip_base.conf file, it's not supporting the vlan as defined. This is included in the migration analysis as the vlans are specified in my virtual server, but the mechanisms for doing so are different in Next. (Note: I don't fully grok this change yet. This article will be updated once I have confidence I'm communicating the functionality accurately.) And finally, from the bigip.conf file, there are few areas of concern, shown in the animated gif below. Standalone bot-defense is not a thing in BIG-IP Next, it's part of the overall policy, so that object is not supported. Also not supported yet are local traffic policies and cipher groups. Note that even though these objects aren't supported, I can still migrate the application, and it should "just work." I guess we'll see later in this article, right? :) At this point, select the </> Preview AS3 and copy that to a file. We'll compare that to the classic BIG-IP version of AS3 in a later section. Add an Application Service After closing the AS3 preview, select the application again and click Add. Click Next For this particular application, we need a couple shared objects: the certificate/key pair for the SSL profile and the WAF policy. Click Import. After those are imported, click the numbered icon (2 in my case) under the Shared Objects column, which will open a listing of those objects that you imported. Review the objects (optional) and click Exit. At this step, if your existing application migration is accurate to the object level, you can deploy to an instance directly. But I have some changes to make to the IPs so I'm going to deploy as a draft instead. After seeing that my deployment was successfully deployed as a draft in Central Manager, I click Finish. Update the Draft and Deploy In My Application Services, click the application we just migrated. Here we can tweak the AS3 declaration. I need to update the vlan as my vlan.br1 from my TMOS BIG-IP system is not defined on my Next instance. I also have different client/server address ranges, so I updated the virtual server and pool member addresses as well. You will likely want to change your application name from the generically-migrated "application_5" but I left it as is for this exercise. Once I completed those changes, I clicked Save & Deploy. I was then asked to select an instance to deploy the application server. I only have one currently, so I selected that. This failed due to my vlan configuration. As I mentioned during the migration process, I don't yet fully grok the vlan referencing requirements in Next, so this is a point for me to be educated on and follow up with updates here in this article. Instead, I removed the allowVlans attributed altogether (after another attempt) and then clicked Save & Deploy again and (after re-selecting the deploy location as shown above) found success. Clicking on the application, you get a visual representation of the application objects. Testing and Observing the Migrated Application Now that we have an honest to goodness deployed application on BIG-IP Next (WOO HOO!!) let's test it to make sure things are working as expected. I have a ubuntu test server with connections into my external and internal traffic networks for my Next instance so it can be the client (curl) and the server (NGINX). First, a request that should work: curl -sk https://10.0.2.50/ <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> Huzzah! That's a successful test. I ran a simple bash script with repetitive wget calls to push just a little load to populate the instance traffic graph: Now let's test the WAF policy by sending some nefarious traffic: curl -sk --config requests.txt https://10.0.2.50/ <html> <head> <title>Request Rejected</title> </head> <body>The requested URL was rejected. Please consult with your administrator.<br><br> Your support ID is: 16177875355615369771<br><br> <a href='javascript:history.back();'>[Go Back]</a> </body> </html> Sweet! Exactly what we wanted to see. Now let's take a look at the WAF Dashboard for blocks. Ok, that's a wrap on migrating the application. Functionally, it is a success! Comparing BIG-IP classic AS3 with BIG-IP Next AS3 If you are moving from classic BIG-IP configuration to BIG-IP Next, you likely will not have any context for comparing AS3 and so you might miss that some of the features you configured in classic are not present in Next. Some of those features aren't there at all yet, and some of them are just not exposed yet. Under the hood, TMM is still TMM with BIG-IP Next, and all of that core functionality is there, it's just a matter of prioritizing what gets exposed and tested and ready to support. Despite a myriad of features in classic BIG-IP, a surprising number of features went either unused or under-used and maintaining support for those will depend on future use requirements. Anyway, one way to build context for AS3 is to useVisual Studio Code and the F5 Extension to take your classic configuration and convert that to AS3 declarations with the AS3 configuration converter. In this section, I'm going to look at a few snippets to compare between classic and Next. Declaration Header The header for classic is a essentially a wrapper (lines 2-4) that isn't necessary in Next at all. That's because in classic, AS3 is not the only declaration class, you also have declarative onboarding and telemetry streaming. Classic: { "$schema": "https://raw.githubusercontent.com/F5Networks/f5-appsvcs-extension/master/schema/latest/as3-schema.json", "class": "AS3", "declaration": { "class": "ADC", "schemaVersion": "3.37.0", "id": "urn:uuid:4339ea7d-094b-4950-b029-ac6344b03a2b", "label": "Converted Declaration", } } Next: { "class": "ADC", "schemaVersion": "3.0.0", "id": "urn:uuid:715aa8d8-c2b0-4890-9e77-5f6131ee9efd", "label": "Converted Declaration", } Profiles One thing to keep in mind with migration is that the migration assistant currently provides detailed analysis to the class level, not the class attribute level. This means that some of the attributes that are supported in classic that are not supported in Next will fly under the radar and be removed with no notification. There is work underway in this regard, but you'll need to evaluate each of your applications as you migrate and plan accordingly. For the app I migrated here, this was evident in the following profiles. ClientSSL Here, the cipher groups and rules from classic are not yet available, and the ability to establish only TLSv1.3 seems to not be configurable at this time. Classic: "cssl.TestSuite": { "certificates": [ { "certificate": "foo.acmelabs.com" } ], "cipherGroup": { "use": "cg_TLSv1.3" }, "class": "TLS_Server", "tls1_0Enabled": true, "tls1_1Enabled": true, "tls1_2Enabled": true, "tls1_3Enabled": true, "singleUseDhEnabled": false, "insertEmptyFragmentsEnabled": false }, Next: "cssl.TestSuite": { "authenticationFrequency": "one-time", "certificates": [ { "certificate": "/tenant87f7bd9913a51/application_5/foo.acmelabs.com" } ], "class": "TLS_Server" }, TCP In the TCP profile, the most notable changes are the loss of QoS settings and the ability to select the congestion control algorithm. Classic: "customTCP": { "congestionControl": "bbr", "idleTimeout": 600, "ipTosToClient": "pass-through", "keepAliveInterval": 2100, "pktLossIgnoreBurst": 3, "pktLossIgnoreRate": 10, "proxyOptions": true, "class": "TCP_Profile" } Next: "customTCP": { "idleTimeout": 600, "pktLossIgnoreBurst": 3, "pktLossIgnoreRate": 10, "proxyBufferHigh": 262144, "proxyBufferLow": 196608, "proxyOptions": true, "sendBufferSize": 262144, "class": "TCP_Profile" }, HTTP In my HTTP profile, it seems I lost all my personally-selected options, such that I'd likely be fine with the default profile. Also, since I'm using the WAF, I can manage the allowed request methods there, and whereas I can't auto-insert strict transport security in the profile directly yet, I can manage that in an iRule as well, so I do have a path to workarounds for both cases. Classic: "customHTTP": { "knownMethods": [ "PATCH", "DELETE", "GET", "POST", "PUT" ], "maxHeaderCount": 32, "maxHeaderSize": 16384, "hstsInsert": true, "xForwardedFor": true, "proxyType": "reverse", "class": "HTTP_Profile" }, Next: "customHTTP": { "requestChunking": "sustain", "responseChunking": "sustain", "class": "HTTP_Profile" } Final Thoughts I point out the differences in my before and after to show a complete picture of the migration process. Some things changed, some went away, but the bottom line is I have a working application service. Before working on this article, I've done a migration in a couple step-by-step controlled labs and have played with but not finished deploying a working, tested, functional application in my own lab. Don't make that same mistake. Get your classic configurations migrated ASAP even if only as a draft in Central Manager, so you can start to evaluate and analyze What work you have on your end to tweak and tune where features have changed Where you need to start engaging your account team to inquire about your MUST HAVE features that may or may not be scoped currently. Next time out, we'll take a look at creating a net-new application service. Until then, stay active out there community and start digging into BIG-IP Next!
Creating a Docker Container to Run AS3 Declarations
This guide will take you through some very basic docker, Python, and F5 AS3 configuration to create a single-function container that will update a pre-determined BIG-IP using an AS3 declaration stored on Github. While it’s far from production ready, it might serve as a basis for more complex configurations, plus it illustrates nicely some technology you can use to automate BIG-IP configuration using AS3, Python and containers. I'm starting with a running BIG-IP - in this case a VE running on theGoogle Cloud Platform, with the AS3 wroker installed and provisioned, plus a couple of webservers listening on different ports. First we’re going to need a host running docker. Fire up an instance in on the platform of your choice – in this example I’m using Ubuntu 18.04 LTS on the Google Cloud platform – that’s purely from familiarity – anything that can run Docker will do. The install process is well documented but looks a bit like this: $ sudo apt-get update $ sudo apt-get install \ apt-transport-https \ ca-certificates \ curl \ gnupg-agent \ software-properties-common $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - $sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" $sudo apt-get update $ sudoapt-get install docker-ce docker-ce-cli containerd.io It's worth adding your user to the docker group to avoid repeadly forgetting to type sudo (or is that just me?) $ sudo usermod -aGdocker$USER Next let's test it's all working: $ docker run hello-world Next let’s take a look at the AS3 declaration,as you might expect form me by now, it’s the most basic version – a simple HTTP app, with two pool members. The beauty of the AS3 model, of course, is that it doesn’t matter how complex your declaration is, the implementation is always the same. So you could take a much more involved declaration and just by changing the file the python script uses, get a more complex configuration. { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.0.0", "id": "urn:uuid:33045210-3ab8-4636-9b2a-c98d22ab915d", "label": "Sample 1", "remark": "Simple HTTP Service with Round-Robin Load Balancing", "Sample_01": { "class": "Tenant", "A1": { "class": "Application", "template": "http", "serviceMain": { "class": "Service_HTTP", "virtualAddresses": [ "10.138.0.4" ], "pool": "web_pool" }, "web_pool": { "class": "Pool", "monitors": [ "http" ], "members": [ { "servicePort": 8080, "serverAddresses": [ "10.138.0.3" ] }, { "servicePort": 8081, "serverAddresses": [ "10.138.0.3" ] } ] } } } } } Now we need some python code to fire up our request. The code below is absolutely a minimum viable set that’s been written for simplicity and clarity and does minimal error checking. There are more ways to improve it that lines of code in it, but it will get you started. #Python Code to run an as3 declaration # import requests import os from requests.auth import HTTPBasicAuth # Get rid of annoying insecure requests waring from requests.packages.urllib3.exceptions import InsecureRequestWarning requests.packages.urllib3.disable_warnings(InsecureRequestWarning) # Declaration location GITDRC = 'https://raw.githubusercontent.com/RuncibleSpoon/as3/master/declarations/payload.json' IP = '10.138.0.4' PORT = '8443' USER = os.environ['XUSER'] PASS = os.environ['XPASS'] URLBASE = 'https://' + IP + ':' + PORT TESTPATH = '/mgmt/shared/appsvcs/info' AS3PATH = '/mgmt/shared/appsvcs/declare' print("########### Fetching Declaration ###########") d = requests.get(GITDRC) # Check we have connectivity and AS3 is installed print('########### Checking that AS3 is running on ', IP ,' #########') url = URLBASE + TESTPATH r = requests.get(url, auth=HTTPBasicAuth(USER, PASS), verify=False) if r.status_code == 200: data = r.json() if data["version"]: print('AS3 version is ', data["version"]) print('########## Runnig Declaration #############') url = URLBASE + AS3PATH headers = { 'content-type': 'application/json', 'accept': 'application/json' } r = requests.post(url, auth=HTTPBasicAuth(USER, PASS), verify=False, data=d.text, headers=headers) print('Status Code:', r.status_code,'\n', r.text) else: print('AS3 test to ',IP, 'failed: ', r.text) This simple python code will pull down an S3 declaration from GitHub using the 'requests' Python library, and the GITDRC variable, connect to a specific BIG-IP, test it’s running AS3 (see here for AS3 setup instructions), and then apply the declaration. It will give you some tracing output, but that’s about it. There are couple of things to note about IP’s, users, and passwords: IP = '10.138.0.4' PORT = '8443' USER = os.environ['XUSER'] PASS = os.environ['XPASS' As you can see, I’ve set the IP and port statically and the username and passwords are pulled in from environment variables in the container. We’ll talk more about the environment variables below, but this is more a way to illustrate your options than design advice. Now we need to build a container to run it in. Containers are relatively easy to build with just a Dockerfile and a few more test files in a directory. Here's the docker file: FROM python:3 WORKDIR /usr/src/app ARG Username=admin ENV XUSER=$Username ARG Password=admin ENV XPASS=$Password # line bleow is not actually used - see comments - but oy probably is a better way ARG DecURL=https://raw.githubusercontent.com/RuncibleSpoon/as3/master/declarations/payload.json ENV Declaration=$DecURL COPY requirements.txt ./ RUN pip install --no-cache-dir -r requirements.txt COPY . . ENTRYPOINT [ "python", "./as3.py" ] You can see a couple of ARG and ENV statements , these simple set the environment variables that we’re (somewhat arbitrarily) using in the python script. Further more we’re going to override them in then build command later. It’s worth noting this isn’t a way to obfuscate passwords, they are exposed by a simple $ docker image history command that will expose all sorts of things about the build of the container, including the environment variables passes to it. This can be overcome by a multi-stage build – but proper secret management is something you should explore – and comment below if you’d like some examples. What’s this requirements.txt file mentioned in the Dockerfile it’s just a manifest to the install of the python package we need: # This file is used by pip to install required python packages # Usage: pip install -r requirements.txt # requests package requests==2.21.0 With our Dockerfile, requirements.txt and as3.py files in a directory we're ready to build a container - in this case I'm going to pass some environment variables into the build to be incorporated in the image - and replace the ones we have set in the Dockerfile: $ export XUSER=admin $ export XPASS=admin Build the container (the -t flag names and tags your container, more of which later): $ docker build -t runciblespoon/as3_python:A--build-arg Username=$XUSER --build-arg Password=$XPASS . The first time you do this there will be some lag as files for the python3 source containerare downloaded and cached, but once it has run you should be able to see your image: $ docker image list REPOSITORY TAG IMAGE ID CREATED SIZE runciblespoon/as3_python A 819dfeaad5eb About a minute ago 936MB python 3 954987809e63 42 hours ago 929MB hello-world latest fce289e99eb9 3 months ago 1.84kB Now we are ready to run the container maybe a good time for a 'nothing up my sleeve' moment - here is the state of the BIG-IP before hand Now let's run the container from our image.The -tty flag attached a pseudo terminal for output and --rm deletes the container afterwards: $ docker run --tty --rm runciblespoon/as3_python:A ########### Fetching Declaration ########### ########### Checking that AS3 is running on 10.138.0.4 ######### AS3 version is 3.10.0 ########## Runing Declaration ############# Status Code: 200 {"results":[{"message":"success","lineCount":23,"code":200,"host":"localhost","tenant":"Sample_01","runTime":929}],"declaration":{"class":"ADC","schemaVersion":"3.0.0","id":"urn:uuid:33045210-3ab8-4636-9b2a-c98d22ab915d","label":"Sample 1","remark":"Simple HTTP Service with Round-Robin Load Balancing","Sample_01":{"class":"Tenant","A1":{"class":"Application","template":"http","serviceMain":{"class":"Service_HTTP","virtualAddresses":["10.138.0.4"],"pool":"web_pool"},"web_pool":{"class":"Pool","monitors":["http"],"members":[{"servicePort":8080,"serverAddresses":["10.138.0.3"]},{"servicePort":8081,"serverAddresses":["10.138.0.3"]}]}}},"updateMode":"selective","controls":{"archiveTimestamp":"2019-04-26T18:40:56.861Z"}}} Success, by the looks of things. Let's check the BIG-IP: running our container has pulled down the AS3 declaration, and applied it to the BIG-IP. This same container can now be run repeatedly - and only the AS3 declaration stored in git (or anywhere else your container can get it from) needs to change. So now you have this container running locally you might want ot put it somewhere. Docker hub is a good choice and lets you create one private repository for free. Remember this container image has credentials, so keep it safe and private. Now the reason for the -t runciblespoon/as3_python:A flag earlier. My docker hub user is "runciblespoon" and my private repository is as3_python. So now all i need ot do is login to Docker Hub and push my image there: $ docker login $ docker push runciblespoon/as3_python:B Now I can go to any other host that runs Docker, login to Docker hub and run my container: $ docker login $ docker run --tty --rm runciblespoon/as3_python:A Unable to find image 'runciblespoon/as3_python:A' locally B: Pulling from runciblespoon/as3_python ... ########### Fetching Declaration ########### Docker will pull down my container form my private repo and run it, using the AS3 declaration I've specified. If I want to change my config, I just change the declaration and run it again. Hopefully this article gives you a starting point to develop your own containers, python scripts, or AS3 declarations, I'd be interested in what more you would like to see, please ask away in the comments section.
AS3 Best Practice
Introduction AS3 is a declarative API that uses JSON key-value pairs to describe a BIG-IP configuration. From virtual IP to virtual server, to the members, pools, and nodes required, AS3 provides a simple, readable format in which to describe a configuration. Once you've got the configuration, all that's needed is to POST it to the BIG-IP, where the AS3 extension will happily accept it and execute the commands necessary to turn it into a fully functional, deployed BIG-IP configuration. If you are new to AS3, start reading the following references: Products - Automation and orchestration toolchain(f5.com; Product information) Application Services 3 Extension Documentation(clouddocs; API documentation and guides) F5 Application Services 3 Extension(AS3) (GitHub; Source repository) This article describes some considerations in order to efficiently deploy the AS3 configurations. Architecture In the TMOS space, the services that AS3 provides are processed by a daemon named 'restnoded'. It relies on the existing BIG-IP framework for deploying declarations. The framework consists of httpd, restjavad and icrd_child as depicted below (the numbers in parenthesis are listening TCP port numbers). These processes are also used by other services. For example, restjavad is a gateway for all the iControl REST requests, and is used by a number of services on BIG-IP and BIG-IQ. When an interaction between any of the processes fails, AS3 operation fails. The failures stem from lack of resources, timeouts, data exceeding predefined thresholds, resource contention among the services, and more. In order to complete AS3 operations successfully, it is advised to follow the Best Practice outlined below. Best Practice Your single source of truth is your declaration Refrain from overwriting the AS3-deployed BIG-IP configurations by the other means such as TMSH, GUI or iControl REST calls. Since you started to use the AS3 declarative model, the source of truth for your device's configurations is in your declaration, not the BIG-IP configuration files. Although AS3 tries to weigh BIG-IP locally stored configurations as much as it can do, discrepancy between the declaration and the current configuration on BIG-IP may cause the AS3 to perform less efficiently or error unexpectedly. When you wish to change a section of a tenant (e.g., pool name change), modify the declaration and submit it. Keep the number of applications in one tenant to a minimum AS3 processes each tenant separately.Having too many applications (virtual servers) in a single tenant (partition) results in a lengthy poll when determining the current configuration. In extreme cases (thousands of virtuals), the action may time out. When you want to deploy a thousand or more applications on a single device, consider chunking the work for AS3 by spreading the applications across multiple tenants (say, 100 applications per tenant). AS3 tenant access behavior behaves as BIG-IP partition behavior.A non-Common partition virtual cannot gain access to another partition's pool, and in the same way, an AS3 application does not have access to a pool or profile in another tenant.In order to share configuration across tenants, AS3 allows configuration of the "Shared" application within the "Common" tenant.AS3 avoids race conditions while configuring /Common/Shared by processing additions first and deletions last, as shown below.This dual process may cause some additional delay in declaration handling. Overwrite rather than patching (POSTing is a more efficient practice than PATCHing) AS3 is a stateless machine and is idempotent. It polls BIG-IP for its full configuration, performs a current-vs-desired state comparison, and generates an optimal set of REST calls to fill the differences.When the initial state of BIG-IP is blank, the poll time is negligible.This is why initial configuration with AS3 is often quicker than subsequent changes, especially when the tenant contains a large number of applications. AS3 provides the means to partially modify using PATCH (seeAS3 API Methods Details), but do not expect PATCH changes to be performant.AS3 processes each PATCH by (1) performing a GET to obtain the last declaration, (2) patching that declaration, and (3) POSTing the entire declaration to itself.A PATCH of one pool member is therefore slower than a POST of your entire tenant configuration.If you decide to use PATCH,make sure that the tenant configuration is a manageable size. Note: Using PATCH to make a surgical change is convenient, but using PATCH over POST breaks the declarative model. Your declaration should be your single source of truth.If you include PATCH, the source of truth becomes "POST this file, then apply one or more PATCH declarations." Get the latest version AS3 is evolving rapidly with new features that customers have been wishing for along with fixes for known issues. Visitthe AS3 section of the F5 Networks Github.Issuessection shows what features and fixes have been incorporated. For BIG-IQ, check K54909607: BIG-IQ Centralized Management compatibility with F5 Application Services 3 Extension and F5 Declarative Onboarding for compatibilities with BIG-IQ versions before installation. Use administrator Use a user with the administrator role when you submit your declaration to a target BIG-IP device. Your may find your role insufficient to manipulate BIG-IP objects that are included in your declaration. Even one authorized item will cause the entire operation to fail and role back. See the following articles for more on BIG-IP user and role. Manual Chapter : User Roles (12.x) Manual Chapter : User Roles (13.x) Manual Chapter : User Roles (14.x) Prerequisites and Requirements(clouddocs AS3 document) Use Basic Authentication for a large declaration You can choose either Basic Authentication (HTTP Authorization header) or Token-Based Authentication (F5 proprietary X-F5-Auth-Token) for accessing BIG-IP. While the Basic Authentication can be used any time, a token obtained for the Token-Based Authentication expires after 1,200 seconds (20 minutes). While AS3 does re-request a new token upon expiry, it requires time to perform the operation, which may cause AS3 to slow down. Also, the number of tokens for a user is limited to 100 (since 13.1), hence if you happen to have other iControl REST players (such as BIG-IQ or your custom iControl REST scripts) using the Token-Based Authentication for the same user, AS3 may not be able to obtain the next token, and your request will fail. See the following articles for more on the Token-Based Authentication. Demystifying iControl REST Part 6: Token-Based Authentication(DevCentral article). iControl REST Authentication Token Management(DevCentral article) Authentication and Authorization(clouddocs AS3 document) Choose the best window for deployment AS3 (restnoded daemon) is a Control Plane process. It competes against other Control Plane processes such as monpd and iRules LX (node.js) for CPU/memory resources. AS3 uses the iControl REST framework for manipulating the BIG-IP resources. This implies that its operation is impacted by any processes that use httpd (e.g., GUI), restjavad, icrd_child and mcpd. If you have resource-hungry processes that run periodically (e.g., avrd), you may want to run your AS3 declaration during some other time window. See the following K articles for alist of processes K89999342 BIG-IP Daemons (12.x) K05645522BIG-IP Daemons (v13.x) K67197865BIG-IP Daemons (v14.x) K14020: BIG-IP ASM daemons (11.x - 15.x) K14462: Overview of BIG-IP AAM daemons (11.x - 15.x) Workarounds If you experience issues such as timeout on restjavad, it is possible that your AS3 operation had resource issues. After reviewing the Best Practice above but still unable to alleviate the problem, you may be able to temporarily fix it by applying the following tactics. Increase the restjavad memory allocation The memory size of restjavad can be increased by the following tmsh sys db commands tmsh modify sys db provision.extramb value <value> tmsh modify sys db restjavad.useextramb value true The provision.extramb db key changes the maximum Java heap memory to (192 + <value> * 8 / 10) MB. The default value is 0. After changing the memory size, you need to restart restjavad. tmsh restart sys service restjavad See the following article for more on the memory allocation: K26427018: Overview of Management provisioning Increase a number of icrd_child processes restjavad spawns a number of icrd_child processes depending on the load. The maximum number of icrd_child processes can be configured from /etc/icrd.conf. Please consult F5 Support for details. See the following article for more on the icrd_child process verbosity: K96840770: Configuring the log verbosity for iControl REST API related to icrd_child Decrease the verbosity levels of restjavad and icrd_child Writing log messages to the file system is not exactly free of charge. Writing unnecessarily large amount of messages to files would increase the I/O wait, hence results in slowness of processes. If you have changed the verbosity levels of restjavad and/or icrd_child, consider rolling back the default levels. See the following article for methods to change verbosity level: K15436: Configuring the verbosity for restjavad logs on the BIG-IP systemAdvanced WAF v16.0 - Declarative API
Since v15.1 (in draft), F5® BIG-IP® Advanced WAF™ canimport Declarative WAF policy in JSON format. The F5® BIG-IP® Advanced Web Application Firewall (Advanced WAF) security policies can be deployed using the declarative JSON format, facilitating easy integration into a CI/CD pipeline. The declarative policies are extracted from a source control system, for example Git, and imported into the BIG-IP. Using the provided declarative policy templates, you can modify the necessary parameters, save the JSON file, and import the updated security policy into your BIG-IP devices. The declarative policy copies the content of the template and adds the adjustments and modifications on to it. The templates therefore allow you to concentrate only on the specific settings that need to be adapted for the specific application that the policy protects. ThisDeclarative WAF JSON policyis similar toNGINX App Protect policy. You can find more information on theDeclarative Policyhere : NAP :https://docs.nginx.com/nginx-app-protect/policy/ Adv. WAF :https://techdocs.f5.com/en-us/bigip-15-1-0/big-ip-declarative-security-policy.html Audience This guide is written for IT professionals who need to automate their WAF policy and are familiar with Advanced WAF configuration. These IT professionals can fill a variety of roles: SecOps deploying and maintaining WAF policy in Advanced WAF DevOps deploying applications in modern environment and willing to integrate Advanced WAF in their CI/CD pipeline F5 partners who sell technology or create implementation documentation This article covershow to PUSH/PULL a declarative WAF policy in Advanced WAF: With Postman With AS3 Table of contents Upload Policy in BIG-IP Check the import Apply the policy OpenAPI Spec File import AS3 declaration CI/CD integration Find the Policy-ID Update an existing policy Video demonstration First of all, you need aJSON WAF policy, as below : { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false } } } 1. Upload Policy in BIG-IP There are 2 options to upload a JSON file into the BIG-IP: 1.1 Either youPUSHthe file into the BIG-IP and you IMPORT IT OR 1.2 the BIG-IPPULLthe file froma repository (and the IMPORT is included)<- BEST option 1.1PUSH JSON file into the BIG-IP The call is below. As you can notice, it requires a 'Content-Range' header. And the value is 0-(filesize-1)/filesize. In the example below, the file size is 662 bytes. This is not easy to integrate in a CICD pipeline, so we created the PULL method instead of the PUSH (in v16.0) curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/file-transfer/uploads/policy-api.json' \ --header 'Content-Range: 0-661/662' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --header 'Content-Type: application/json' \ --data-binary '@/C:/Users/user/Desktop/policy-api.json' At this stage,the policy is still a filein the BIG-IP file system. We need toimportit into Adv. WAF. To do so, the next call is required. This call import the file "policy-api.json" uploaded previously. AnCREATEthe policy /Common/policy-api-arcadia curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/javascript' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "filename":"policy-api.json", "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 1.2PULL JSON file from a repository Here, theJSON file is hosted somewhere(in Gitlab or Github ...). And theBIG-IP will pull it. The call is below. As you can notice, the call refers to the remote repo and the body is a JSON payload. Just change the link value with your JSON policy URL. With one call, the policy isPULLEDandIMPORTED. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }' Asecond versionof this call exists, and refer to the fullPath of the policy.This will allow you to update the policy, from a second version of the JSON file, easily.One call for the creation and the update. As you can notice below, we add the"policy":"fullPath" directive. The value of the "fullPath" is thepartitionand thename of the policyset in the JSON policy file. This method is VERY USEFUL for CI/CD integrations. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' 2. Check the IMPORT Check if the IMPORT worked. To do so, run the next call. curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ You should see a 200 OK, with the content below (truncated in this example). Please notice the"status":"COMPLETED". { "kind": "tm:asm:tasks:import-policy:import-policy-taskcollectionstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy?ver=16.0.0", "totalItems": 11, "items": [ { "isBase64": false, "executionStartTime": "2020-07-21T15:50:22Z", "status": "COMPLETED", "lastUpdateMicros": 1.595346627e+15, "getPolicyAttributesOnly": false, ... From now, your policy is imported and created in the BIG-IP. You can assign it to a VS as usual (Imperative Call or AS3 Call).But in the next session, I will show you how to create a Service with AS3 including the WAF policy. 3. APPLY the policy As you may know, a WAF policy needs to be applied after each change. This is the call. curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/apply-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{"policy":{"fullPath":"/Common/policy-api-arcadia"}}' 4. OpenAPI spec file IMPORT As you know,Adv. WAF supports OpenAPI spec (2.0 and 3.0). Now, with the declarative WAF, we can import the OAS file as well. The BEST solution, is toPULL the OAS filefrom a repo. And in most of the customer' projects, it will be the case. In the example below, the OAS file is hosted in SwaggerHub(Github for Swagger files). But the file could reside in a private Gitlab repo for instance. The URL of the projectis :https://app.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 The URL of the OAS file is :https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3 This swagger file (OpenAPI 3.0 Spec file) includes all the application URL and parameters. What's more, it includes the documentation (for NGINX APIm Dev Portal). Now, it ispretty easy to create a WAF JSON Policy with API Security template, referring to the OAS file. Below, you can notice thenew section "open-api-files"with the link reference to SwaggerHub. And thenew templatePOLICY_TEMPLATE_API_SECURITY. Now, when I upload / import and apply the policy, Adv. WAF will download the OAS file from SwaggerHub and create the policy based on API_Security template. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "https://api.swaggerhub.com/apis/F5EMEASSA/Arcadia-OAS3/1.0.0-oas3" } ] } } 5. AS3 declaration Now, it is time to learn how we cando all of these steps in one call with AS3(3.18 minimum). The documentation is here :https://clouddocs.f5.com/products/extensions/f5-appsvcs-extension/latest/declarations/application-security.html?highlight=waf_policy#virtual-service-referencing-an-external-security-policy With thisAS3 declaration, we: Import the WAF policy from a external repo Import the Swagger file (if the WAF policy refers to an OAS file) from an external repo Create the service { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "API": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["10.1.10.27"], "redirect80": false, "pool": "pool_NGINX_API_AS3", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://10.1.20.4/root/as3-waf-api/-/raw/master/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API_AS3": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["10.1.20.9"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } 6. CI/CID integration As you can notice, it is very easy to create a service with a WAF policy pulled from an external repo. So, it is easy to integrate these calls (or the AS3 call) into a CI/CD pipeline. Below, an Ansible playbook example. This playbook run the AS3 call above. That's it :) --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "admin" my_password: "admin" bigip: "10.1.1.12" tasks: - name: Deploy AS3 WebApp uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic YWRtaW46YWRtaW4=" body: "{{ lookup('file','as3.json') }}" body_format: json validate_certs: no status_code: 200 7. FIND the Policy-ID When the policy is created, a Policy-ID is assigned. By default, this ID doesn't appearanywhere. Neither in the GUI, nor in the response after the creation. You have to calculate it or ask for it. This ID is required for several actions in a CI/CD pipeline. 7.1 Calculate the Policy-ID Wecreated this python script to calculate the Policy-ID. It is an hash from the Policy name (including the partition). For the previous created policy named"/Common/policy-api-arcadia",the policy ID is"Ar5wrwmFRroUYsMA6DuxlQ" Paste this python codein a newwaf-policy-id.pyfile, and run the commandpython waf-policy-id.py "/Common/policy-api-arcadia" Outcome will beThe Policy-ID for /Common/policy-api-arcadia is: Ar5wrwmFRroUYsMA6DuxlQ #!/usr/bin/python from hashlib import md5 import base64 import sys pname = sys.argv[1] print 'The Policy-ID for', sys.argv[1], 'is:', base64.b64encode(md5(pname.encode()).digest()).replace("=", "") 7.2 Retrieve the Policy-ID and fullPath with a REST API call Make this call below, and you will see in the response, all the policy creations. Find yours and collect thePolicyReference directive.The Policy-ID is in the link value "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0" You can see as well, at the end of the definition, the "fileReference"referring to the JSON file pulled by the BIG-IP. And please notice the"fullPath", required if you want to update your policy curl --location --request GET 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Range: 0-601/601' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ { "isBase64": false, "executionStartTime": "2020-07-22T11:23:42Z", "status": "COMPLETED", "lastUpdateMicros": 1.595417027e+15, "getPolicyAttributesOnly": false, "kind": "tm:asm:tasks:import-policy:import-policy-taskstate", "selfLink": "https://localhost/mgmt/tm/asm/tasks/import-policy/B45J0ySjSJ9y9fsPZ2JNvA?ver=16.0.0", "filename": "", "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "endTime": "2020-07-22T11:23:47Z", "startTime": "2020-07-22T11:23:42Z", "id": "B45J0ySjSJ9y9fsPZ2JNvA", "retainInheritanceSettings": false, "result": { "policyReference": { "link": "https://localhost/mgmt/tm/asm/policies/Ar5wrwmFRroUYsMA6DuxlQ?ver=16.0.0", "fullPath": "/Common/policy-api-arcadia" }, "message": "The operation was completed successfully. The security policy name is '/Common/policy-api-arcadia'. " }, "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" } }, 8 UPDATE an existing policy It is pretty easy to update the WAF policy from a new JSON file version. To do so, collect from the previous call7.2 Retrieve the Policy-ID and fullPath with a REST API callthe"Policy" and"fullPath"directive. This is the path of the Policy in the BIG-IP. Then run the call below, same as1.2 PULL JSON file from a repository,but add thePolicy and fullPath directives Don't forget to APPLY this new version of the policy3. APPLY the policy curl --location --request POST 'https://10.1.1.12/mgmt/tm/asm/tasks/import-policy/' \ --header 'Content-Type: application/json' \ --header 'Authorization: Basic YWRtaW46YWRtaW4=' \ --data-raw '{ "fileReference": { "link": "http://10.1.20.4/root/as3-waf/-/raw/master/policy-api.json" }, "policy": { "fullPath":"/Common/policy-api-arcadia" } }' TIP : this call, above, can be used in place of the FIRST call when we created the policy "1.2PULL JSON file from a repository". But be careful, the fullPath is the name set in the JSON policy file. The 2 values need to match: "name": "policy-api-arcadia" in the JSON Policy file pulled by the BIG-IP "policy":"fullPath" in the POST call 9 Video demonstration In order to help you to understand how it looks with the BIG-IP, I created this video covering 4 topics explained in this article : The JSON WAF policy Pull the policy from a remote repository Update the WAF policy with a new version of the declarative JSON file Deploy a full service with AS3 and Declarative WAF policy At the end of this video, you will be able to adapt the REST Declarative API calls to your infrastructure, in order to deploy protected services with your CI/CD pipelines. Direct link to the video on DevCentral YouTube channel : https://youtu.be/EDvVwlwEFRwDeclarative Advanced WAF policy lifecycle in a CI/CD pipeline
The purpose of this article is to show the configuration used to deploy a declarative Advanced WAF policy to a BIG-IP and automatically configure it to protect an API workload by consuming an OpenAPI file describing the application. For this experiment, a Gitlab CI/CD pipeline was used to deploy an API workload to Kubernetes, configure a declarative Adv. WAF policy to a BIG-IP device and tuning it by incorporating learning suggestions exported from the BIG-IP. Lastly, the F5 WAF tester tool was used to determine and improve the defensive posture of the Adv. WAF policy. Deploying the declarative Advanced WAF policy through a CI/CD pipeline To deploy the Adv. WAF policy, the Gitlab CI/CD pipeline is calling an Ansible playbook that will in turn deploy an AS3 application referencing the Adv.WAF policy from a separate JSON file. This allows the application definition and WAF policy to be managed by 2 different groups, for example NetOps and SecOps, supporting separation of duties. The following Ansible playbook was used; --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxx" tasks: - name: Deploy AS3 API AWAF policy uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxx body: "{{ lookup('file','as3_waf_openapi.json') }}" body_format: json validate_certs: no status_code: 200 The Advanced WAF policy 'as3_waf_openapi.json' was specified as follows: { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "arcadia": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["xxxxx"], "redirect80": false, "pool": "pool_NGINX_API", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://xxxx/root/awaf_openapi/-/raw/master/WAF/ansible/bigip/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["xxxx"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } The AS3 declaration will provision a separate Administrative Partition ('API-Prod') containing a Virtual Server ('VS_API'), an Adv. WAF policy ('Arcadia_WAF_API_policy') and a pool ('pool_NGINX_API'). The Adv.WAF policy being referenced ('policy-api.json') is stored in the same Gitlab repository but can be downloaded from a separate location. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ ] } The declarative Adv.WAF policy is referencing in turn the OpenAPI file ('openapi3-arcadia.yaml') that describes the application being protected. Executing the Ansible playbook results in the AS3 application being deployed, along with the Adv.WAF policy that is automatically configured according to the OpenAPI file. Handling learning suggestions in a CI/CD pipeline The next step in the CI/CD pipeline used for this experiment was to send legitimate traffic using the API and collect the learning suggestions generated by the Adv.WAF policy, which will allow a simple way to customize the WAF policy further for the specific application being protected. The following Ansible playbook was used to retrieve the learning suggestions: --- - hosts: bigip connection: local gather_facts: true vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxxx" tasks: - name: Get all Policy_key/IDs for WAF policies uri: url: 'https://{{ bigip }}/mgmt/tm/asm/policies?$select=name,id' method: GET headers: "Authorization": "Basic xxxxxxxxxxx" validate_certs: no status_code: 200 return_content: yes register: waf_policies - name: Extract Policy_key/ID of Arcadia_WAF_API_policy set_fact: Arcadia_WAF_API_policy_ID="{{ item.id }}" loop: "{{ (waf_policies.content|from_json)['items'] }}" when: item.name == "Arcadia_WAF_API_policy" - name: Export learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxxx" body: "{ \"inline\": \"true\", \"policyReference\": { \"link\": \"https://{{ bigip }}/mgmt/tm/asm/policies/{{ Arcadia_WAF_API_policy_ID }}/\" } }" body_format: json validate_certs: no status_code: - 200 - 201 - 202 - name: Get learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: GET headers: "Authorization": "Basic xxxxxxxxx" validate_certs: no status_code: 200 register: result - name: Print learning suggestions debug: var=result A sample learning suggestions output is shown below: "json": { "items": [ { "endTime": "xxxxxxxxxxxxx", "id": "ZQDaRVecGeqHwAW1LDzZTQ", "inline": true, "kind": "tm:asm:tasks:export-suggestions:export-suggestions-taskstate", "lastUpdateMicros": 1599953296000000.0, "result": { "suggestions": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "Check maximum number of parameters" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "No Host header in HTTP/1.1 request" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable enforcement of policy violation", "entity": { "name": "VIOL_REQUEST_MAX_LENGTH" }, "entityChanges": { "alarm": true, "block": true }, "entityType": "violation" } Incorporating the learning suggestions in the Adv.WAF policy can be done by simple copy&pasting the self-contained learning suggestions blocks into the "modifications" list of the Adv.WAF policy: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } Enhancing Advanced WAF policy posture by using the F5 WAF tester The F5 WAF tester is a tool that generates known attacks and checks the response of the WAF policy. For example, running the F5 WAF tester against a policy that has a "transparent" enforcement mode will cause the tests to fail as the attacks will not be blocked. The F5 WAF tester can suggest possible enhancement of the policy, in this case the change of the enforcement mode. An abbreviated sample output of the F5 WAF Tester: ................................................................ "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" } }, "summary": { "fail": 48, "pass": 0 } Changing the enforcement mode from "transparent" to "blocking" can easily be done by editing the same Adv. WAF policy file: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } A successful run will will be achieved when all the attacks will be blocked. ......................................... "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": true, "reason": "", "support_id": "17540898289451273964" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": true, "reason": "", "support_id": "17540898289451274344" } }, "system": "All systems" } }, "summary": { "fail": 0, "pass": 48 } Conclusion By adding the Advanced WAF policy into a CI/CD pipeline, the WAF policy can be integrated in the lifecycle of the application it is protecting, allowing for continuous testing and improvement of the security posture before it is deployed to production. The flexible model of AS3 and declarative Advanced WAF allows the separation of roles and responsibilities between NetOps and SecOps, while providing an easy way for tuning the policy to the specifics of the application being protected. Links UDF lab environment link. Short instructional video link.
High-level Pathways to Security Visibility
Editor's Note: The F5 Beacon capabilities referenced in this article hosted on F5 Cloud Services are planning a migration to a new SaaS Platform - Check out the latesthere. (10 Minute Read) Introduction In previous articles we identified the elements needed to gain visibility into adaptive application security postures. This entails observing the security configuration (static) and monitoring telemetry (dynamic) coming from different control points (ref. Visibility and Orchestration).We also suggested that security visibility should be integrated in the software development and/or deployment lifecycle as part of a shift-left strategy (ref. Shift-left Security Visibility). Now, we’ll focus on identifying a high-level pathway to achieve application security visibility. First, we need to identify the constraints that frame the effort.We will then identify concrete examples of insertion with F5 technologies. The end-goal is to ensure that you keep close control over the application security by embracing a holistic approach to visibility integrated in the software development/deployment lifecycle. Constraints Inserting security visibility in your enterprise is part of the shift-left strategy (ref. url-to-shift-left-sec-vis.). (https://www.f5.com/company/blog/beyond-visibility-is-operability) In order to be practical, we need to make sure that the pathway adheres to the following guidelines: Friction – The solution should not introduce any friction into the pipeline - For example, the tools used by the DEVOPS and SECOPS teams (e.g. Gitlab, Jenkins) should be the same avoiding gated interdependencies where a change by one group is blocked/delayed by the other. Programmability – The security-centric solutions implemented during the journey need to be highly programmable – This will ensure that the tools adapt to the environment (e.g. services, micro-services), the supporting infrastructure (e.g. cloud, containers), and the application. Automation – Enabling automation is key. This can be achieved by ensuring the tools deployed can be automatically configured without intervention as part of a pipeline.One way to ensure this is to leverage declarative application programing interfaces (API) (link-to-f5-declarative-interface) Scalability – Applications can span across infrastructure that is infinitely scalable like public cloud, across availability zones and geographies.This requires that any solution that is deployed to secure/protect applications and workloads be able to scale.To scale horizontally, the solution can be implemented across multiple workloads in multiple instances.To scale vertically, the solution should be able to handle increasing amounts of traffic in single/few instances. Transparency – From a performance and functionality standpoint, the solutions inserted to gain security visibility cannot impact the application.For example, when a proxy is inserted, it cannot add latency between the client and the workload. It also cannot affect the functionality provided by the workload. Resiliency – Inserting a solution to support your applications security and visibility should be resilient.Any failure of the process providing visibility should be flagged and not affect the application’s/workload’s performance or availability. Visibility Insertion All F5 solutions can be inserted in the application delivery infrastructure to provide security visibility.This comes in the form security-aware proxies.The BIG-IP or NGINX Plus platforms are particularly well-suited for insertion in infrastructure requiring inline low-latency and powerful application security and visibility.Deploying F5 solutions can easily be done observing all the constraints mentioned above. Friction Thanks to the available form factors and programmatic templates provided, implementing BIG-IP or NGINX Plus in the infrastructure is easily achieved using appropriate templates. For example, when working with AWS, a BIG-IP can easily be deployed using a Cloud Formation Template (CFT) found here.From the enterprise git (Gitlab, Github, Bitbucket etc.) repository, BIG-IP can be deployed directly by cloning/forking the F5 repository and integrating with the pipeline (ref. Clouddocs Article). Programmability The BIG-IP Advanced Web Application Firewall (Advanced WAF) configuration is highly programable.The advantage is that the configuration can be stored and or modified easily outside of the BIG-IP. For example, an base policy aimed at protecting against OWASP Top 10 Risks can look like the following: { "policy": { "name": "Complete_OWASP_Top_Ten", "description": "A generic, OWASP Top 10 protection items v1.0", "template": { "name": "POLICY_TEMPLATE_RAPID_DEPLOYMENT" }, "fullPath": "/Common/Complete_OWASP_Top_Ten", "enforcementMode":"transparent", "signature-settings":{ "signatureStaging": false, "minimumAccuracyForAutoAddedSignatures": "high" }, "protocolIndependent": true, "caseInsensitive": true, "general": { "trustXff": true }, "data-guard": { "enabled": true }, "policy-builder-server-technologies": { "enableServerTechnologiesDetection": true }, "blocking-settings": { "violations": [ { "alarm": true, "block": true, "description": "ASM Cookie Hijacking", "learn": false, "name": "VIOL_ASM_COOKIE_HIJACKING" }, { "alarm": true, "block": true, "description": "Access from disallowed User/Session/IP/Device ID", "name": "VIOL_SESSION_AWARENESS" }, { "alarm": true, "block": true, "description": "Modified ASM cookie", "learn": true, "name": "VIOL_ASM_COOKIE_MODIFIED" }, { "alarm": true, "block": true, "description": "XML data does not comply with format settings", "learn": true, "name": "VIOL_XML_FORMAT" }, { "name": "VIOL_FILETYPE", "alarm": true, "block": true, "learn": true } ], "evasions": [ { "description": "Bad unescape", "enabled": true, "learn": true }, { "description": "Apache whitespace", "enabled": true, "learn": true }, { "description": "Bare byte decoding", "enabled": true, "learn": true }, { "description": "IIS Unicode codepoints", "enabled": true, "learn": true }, { "description": "IIS backslashes", "enabled": true, "learn": true }, { "description": "%u decoding", "enabled": true, "learn": true }, { "description": "Multiple decoding", "enabled": true, "learn": true, "maxDecodingPasses": 3 }, { "description": "Directory traversals", "enabled": true, "learn": true } ] }, "xml-profiles": [ { "name": "Default", "defenseAttributes": { "allowDTDs": false, "allowExternalReferences": false } } ], "session-tracking": { "sessionTrackingConfiguration": { "enableTrackingSessionHijackingByDeviceId": true } } } } In the example above, aspects of a security policy like evasion techniques, or cookie consumption settings can easily be programmed in the configuration and handled like any other application code for versioning, editing or storing.The standard JSON format can be managed in a Git repository for use in any environment.Documentation for JSON representations of WAF policies can be found here. This is also true for all F5 security platforms including NGINX App Protect or Essential App Protect (ref.NGINX Configuration Guide and EAP API Users Guide). Similarly, configuring BIG-IP to forward security information telemetry to appropriate facilities can be achieved with the use of the Telemetry Streaming framework.For example, in order to configure BIG-IP to send telemetry data to a centralized visibility tool (F5 Beacon, or ELK for example) it can be configured with a declaration like: "class": "Telemetry", "controls": { "class": "Controls", "logLevel": "debug" }, "TS_Poller": { "class": "Telemetry_System_Poller", "interval": 60 }, "TS_Listener": { "class": "Telemetry_Listener", "port": 6514 }, "TS_Consumer": { "class": "Telemetry_Consumer", "type": "Generic_HTTP", "host": "my.visibility-host.url", "protocol": "http", "port": 8888, "path": "/", "method": "POST", "headers": [ { "name": "content-type", "value": "application/json" } ] } } The above declaration identifies the host where it will send telemetry – in this case debug data Scalability, Transparency and Resiliency F5 provides highly scalable, resilient and transparent solutions that can be inserted in any infrastructure to secure and provide visibility into web applications.Discussing these aspects of BIG-IP, NGINX Plus, or NGINX App Protect is beyond the scope of this article.For more information on scalability and high-availability you can refer to Performance of NGINX and NGINX Plus, NGINX App Protect Application Security Testing or BIG-IP Datasheet. Conclusion This article is meant to offer a path to visibility using F5 technology by inserting BIG-IP and configure it to provide application security and generate telemetry to gain visibility into the application's security posture. The aim is for you build a blueprint to systematically watch over your adaptive valuable applications and workloads across your infrastructure.
