F5 and Cisco ACI Essentials - Design guide for a Single Pod APIC cluster
Deployment considerations It is usually an easy decision to have BIG-IP as part of your ACI deployment as BIG-IP is a mature feature rich ADC solution. Where time is spent is nailing down the design and the deployment options for the BIG-IP in the environment. Below we will discuss a few of the most commonly asked questions: SNAT or no SNAT There are various options you can use to insert the BIG-IP into the ACI environment. One way is to use the BIG-IP as a gateway for servers or as a routing next hop for routing instances. Another option is to use Source Network Address Translation (SNAT) on the BIG-IP, however with enabling SNAT the visibility into the real source IP address is lost. If preserving the source IP is a requirement then ACI's Policy-Based Redirect (PBR) can be used to make sure the return traffic goes back to the BIG-IP. BIG-IP redundancy F5 BIG-IP can be deployed in different high-availability modes. The two common BIG-IP deployment modes: active-active and active-standby. Various design considerations, such as endpoint movement during fail-overs, MAC masquerade, source MAC-based forwarding, Link Layer Discovery Protocol (LLDP), and IP aging should also be taken into account for each of the deployment modes. Multi-tenancy Multi-tenancy is supported by both Cisco ACI and F5 BIG-IP in different ways. There are a few ways that multi-tenancy constructs on ACI can be mapped to multi-tenancy on BIG-IP. The constructs revolve around tenants, virtual routing and forwarding (VRF), route domains, and partitions. Multi-tenancy can also be based on the BIG-IP form factor (appliance, virtual edition and/or virtual clustered multiprocessor (vCMP)). Tighter integration Once a design option is selected there are questions around what more can be done from an operational or automation perspective now that we have a BIG-IP and ACI deployment? The F5 ACI ServiceCenter is an application developed on the Cisco ACI App Center platform built for exactly that purpose. It is an integration point between the F5 BIG-IP and Cisco ACI. The application provides an APIC administrator a unified way to manage both L2-L3 and L4-L7 infrastructure. Once day-0 activities are performed and BIG-IP is deployed within the ACI fabric using any of the design options selected for your environment, then the F5 ACI ServiceCenter can be used to handle day-1 and day-2 operations. The day-1 and day-2 operations provided by the application are well suited for both new/greenfield and existing/brownfield deployments of BIG-IP and ACI deployments. The integration is loosely coupled, which allows the F5 ACI ServiceCenter to be installed or uninstalled with no disruption to traffic flow, as well as no effect on the F5 BIG-IP and Cisco ACI configuration. Check here to find out more. All of the above topics and more are discussed in detail here in the single pod white paper.F5 & Cisco ACI Essentials - Dynamic pool sizing using the F5 ACI ServiceCenter
APIC EndPoints and EndPoint Groups When dealing with the Cisco ACI environment you may have wondered about using an Application-Centric Design or a Network-Centric Design. Regardless of the strategy, the ultimate goal is to have an accessible and secure application/workload in the ACI environment. An application is comprised of several servers; each one performing a function for the application (web server, DB server, app server etc.). Each of these servers may be physical or virtual and are treated as endpoints on the ACI fabric. Endpoints are devices connected to the network directly or indirectly. They have an address, attributes and can be physical or virtual. Endpoint examples include servers, virtual machines, network-attached storage, or clients on the Internet. An EPG (EndPoint Group) is an object that contains a collection of endpoints, which can be added to an EPG either dynamically or statically. Take a look at the relationship between different objects on the APIC. Click here for more details. Relationship between Endpoints and Pool members If an application is being served by web servers with IPs having address's in the range 192.168.56.*, for example, then these IP addresses will be presented as an endpoint in an endpoint group (EPG) on the APIC. From the perspective of BIG-IP, these web servers are pool members of a particular pool. The F5 ACI ServiceCenter is an application developed on the Cisco ACI App Center platform designed to run on the APIC controller. It has access to both APIC and BIG-IP and can correlate existing information from both devices to provide a mapping as follows: BIG-IP | APIC ________________________________________________________________________ VIP: Pool: Pool Member(s): Route Domain (RD) |Tenant: Application Profile: End Point group: Virtual Routing and Forwarding (VRF) This gives an administrator a view of how the APIC workload is associated with the BIG-IP and what all applications and virtual IP's are tied to a tenant. Click here for more details on this visibility dashboard and learn more on how and under what situations the dashboard can be helpful. In this article we are going to see how the F5 ACI ServiceCenter can take advantage of the endpoints learned by the ACI fabric to dynamically grow/shrink pool members. Dynamic EndPoint Attach and Detach Let's think back to our application which is say being hosted on 100's of servers, these servers could be added to an APIC EPG statically by a network admin or they could be added dynamically through a vCenter or openstack APIC integration. In either case, these endpoints ALSO need to be added to the BIG-IP where the endpoints can be protected by malicious attacks and/or load-balanced. This can be a very tedious task for a APIC or a BIG-IP administrator. Using the dynamic EndPoint attach and detach feature on the F5 ACI ServiceCenter, this burden can be reduced. The application has the ability to adjust the pool members on the BIG-IP based on the server farm on the APIC. On APIC when an endpoint is attached, it is learned by the fabric and added to a particular tenant, application profile and EPG on the APIC. The F5 ACI ServiceCenter provides the capability to map an EPG on the APIC to a pool on the BIG-IP. The application relies on the attach/detach notifications from the APIC to add/delete the BIG-IP pool-members. There are different ways in which the dynamic mapping can be leveraged using the F5 ACI ServiceCenter based on the L4-L7 configuration: Scenario 1: Declare L4-L7 configuration using F5 ACI ServiceCenter Scenario 2: L4-L7 configuration already exists on the BIG-IP Scenario 3: Use dynamic mapping but do not declare the L4-L7 configuration using the F5 ACI ServiceCenter Scenario 4: Use the F5 ACI ServiceCenter API's to define the mapping along with the L4-L7 configuration Let's take a look at each one of them in detail. Scenario 1: Declare L4-L7 configuration using F5 ACI ServiceCenter Let's assume there is no existing configuration on the BIG-IP, a new application needs to be deployed which is front ended by a VIP/Pool/Pool members. The F5 ACI ServiceCenter provides a UI that can be used to deploy the L4-L7 configuration and create a mapping between Pool <-> EPG. There are two options: Basic mode uses FAST andAdvanced mode uses AS3. Basic mode: Leverage dynamic endpoint attach and detach feature by using the pre-built Service-Discovery template Advanced mode: Leverage dynamic endpoint attach and detach feature by using Manage Endpoint Mappings Scenario 2: L4-L7 configuration already exists on the BIG-IP If L4-L7 configuration using AS3 already exists on the BIG-IP, the F5 ACI ServiceCenter will detect all partitions and application that in compatible with AS3. Configuration for a particular partition/application on BIG-IP can then be updated to create a Pool <-> EPG mapping. However, there is one condition that is the pool can either have static or dynamic members. Thus, if the pool already has existing members, those members will have to be deleted before a dynamic mapping can be created. To maintain the dynamic mapping, any future changes to the L4-L7 configuration on the BIG-IP should be done via the F5 ACI ServiceCenter. Scenario 3: Use dynamic mapping but do not declare the L4-L7 configuration using the F5 ACI ServiceCenter The F5 ACI ServiceCenter can be used just for the dynamic mapping and pool sizing and not for defining the L4-L7 configuration. For this method, the entire AS3 declaration along with the mapping will be directly send to the BIG-IP using AS3. Since the declaration is AS3, the F5 ACI ServiceCenter will automatically detect a Pool <-> EPG mapping which can be viewable from the inventory tab. Step 1: AS3 declaration with Pool <-> EPG mapping posted directly to the BIG-IP (see below for a sample declaration) Step 2: Sync Endpoints Step 3: View Endpoints Scenario 4: Use the F5 ACI ServiceCenter API's to define the mapping along with the L4-L7 configuration Finally, if the UI is not appealing and automation all the way is the goal, then the F5 ACI ServiceCenter has an API call where the mapping as well as the L4-L7 configuration (which was done in Scenario 1) can be completely automated: URI: https://<apic_controller_ip>>/appcenter/F5Networks/F5ACIServiceCenter/updateas3data.json In this scenario, the declaration is being passed to the F5 ACI ServiceCenter through the APIC controller and NOT directly to the BIG-IP. A sample API call Summary Having knowledge on how AS3 works is essential since it is a declarative API, and using it incorrectly can result in incorrect configuration. Any method mentioned above would work, and the decision on which method to use is based on the operational model that works the best in your environment. References Unify Visibility with F5 ACI ServiceCenter in Cisco ACI and F5 BIG-IP Deployments Download F5 ACI ServiceCenter F5 ACI ServiceCenter API documentation F5 AS3 - What does it means -imperative vs declarative F5 AS3 Best Practice Cisco ACI Fabric Endpoint Learning White Paper
iControl REST Fine-Grained Role Based Access Control
Introduction F5's role based access control (RBAC) mechanism allows a BIG-IP administrator to assign appropriate access privileges to the users (see Manual Chapter: User Roles). For example, with the operator role, the user is specifically allowed to enable or disable nodes and pool members. The mechanism is generally the best way to manage users easily and consistently, however, finer access management may be required from time to time: e.g., allow only to view the stats of a predefined set of virtuals. Restricting access is especially important in iControl REST because users can remotely and directly access the system. The existing roles and their access permissions are defined in /mgmt/shared/authz/roles and /mgmt/shared/authz/resource-groups . You can add custom roles with custom permissions for particular users to these resources (iControl REST endpoints). The resources are described in BIG-IQ Systems REST API References, however, they are not exactly easy to follow. To fill the gap, this article describes a method to configure users and roles in a muchfiner manner. Note This method is only applicable to iControl REST: It does not apply to Configuration Utility or ssh. The method relies on the resources for local users (i.e., /mgmt/tm/auth/user and /mgmt/shared/authz/users ). It does not work for remote authentication mechanisms such as RADIUS or Active Directory. Setup In this example, a local user "foo" and a custom role "testRole" are created. The role allows users to GET (list) only the virtual server "vs". The access method and resource are defined in "testResourceGroup", that is referenced from the "testRole". No other operation such as PATCH (modify) or DELETE is permitted. Also, no other resource such as another virtual is allowed. All the REST calls are done using curl: You may want to consider using other methods such as F5 Python SDK . 1. Create a new user The following curl command creates the user "foo". The password is "foo", the description is "Foo Bar", and the role is "operator" for all the partitions. curl -sku admin:<pass> https://<mgmtIP>/mgmt/tm/auth/user -X POST -H "Content-Type: application/json" \ -d '{"name":"foo", "password":"foo", "description":"Foo Bar", "partitionAccess":[ { "name":"all-partitions", "role":"operator"} ] }' You get the following JSON object: { "description": "Foo Bar", "encryptedPassword": "$6$jwL4UUxv$IUrzWGEUsyJaXlOB2oyyTPflHFdvDBXb6f3f/No4KNfQb.V6bpQZBgvxl3KkBDXGtpttej79DDphEGRh8d4iA/", "fullPath": "foo", "generation": 1023, "kind": "tm:auth:user:userstate", "name": "foo", "partitionAccess": [ { "name": "all-partitions", "nameReference": { "link": "https://localhost/mgmt/tm/auth/partition/all-partitions?ver=13.1.1.2" }, "role": "operator" } ], "selfLink": "https://localhost/mgmt/tm/auth/user/satoshi?ver=13.1.1.2" } The role is important. When the access privileges conflict between the role and the fine grained RBAC, the stricter authorization is chosen. For example, if the RBAC is configured to allow PATCH or POST but the user's role is guest (no alteration allowed), the user won't be able to perform these methods. You can create a user from tmsh or Configuration Utility if that's your preferred method. 2. (13.1.0 and later) Remove the user from /mgmt/shared/authz/roles/iControl_REST_API_User From v13.1.0, a newly created local user is automatically added to iControl_REST_API_User , which grants access to iControl REST without any setup. To avoid assigning multiple permissions, you need to remove the user reference from /mgmt/shared/authz/roles/iControl_REST_API_User . First, get the data from /mgmt/shared/authz/roles/iControl_REST_API_User and redirect it to a file. curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/roles/iControl_REST_API_User \ | python -m json.tool > file Edit the file. The file contains a line for the user "foo" like this. "name": "iControl_REST_API_User", "userReferences": [ { "link": "https://localhost/mgmt/shared/authz/users/foo" }, .... Remove the line including the opening and ending curly brackets plus the comma. Save the file. Overwrite the current data by putting the file to the endpoint. curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/roles/iControl_REST_API_User -X PUT -d@file 3. Create a custom resource-group A resource-group consists of a set of resources and methods. In this example, the resource-group is named "testResourceGroup", which allows a role to perform GET request to the resource /mgmt/tm/ltm/virtual/vs . "testResourceGroup" is later used in the custom role. curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/resource-groups -X POST -H "Content-Type: application/json" \ -d '{"name":"testResourceGroup", "resources":[ {"restMethod":"GET", "resourceMask":"/mgmt/tm/ltm/virtual/vs" } ]}' You get the following JSON object. { "id": "fe7a1ebc-3e61-30aa-8a5d-c7721f7c6ce2", "name": "testResourceGroup", "resources": [ { "resourceMask": "/mgmt/tm/ltm/virtual/vs", "restMethod": "GET" } ], "generation": 1, "lastUpdateMicros": 1521682571723849, "kind": "shared:authz:resource-groups:roleresourcegroupstate", "selfLink": "https://localhost/mgmt/shared/authz/resource-groups/fe7a1ebc-3e61-30aa-8a5d-c7721f7c6ce2" } Note that a resource-group entry is keyed by "id", not "name". The id is automatically generated. As you can see, the resources field is a list of multiple objects, each containing the endpoint and the method. To add more permissions to the resource-group, add objects to the list. 4. Create a custom role Create a role "testRole" with the user "foo" and the resource-groups "testResourceGroup". This makes the user to become a member of therole "testRole", hence allows the users to perform GET only to the /mgmt/tm/ltm/virtual/vs . curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/roles -X POST -H "Content-Type: application/json" \ -d '{"name":"testRole", "userReferences":[ {"link":"https://localhost/mgmt/shared/authz/users/foo"} ], "resourceGroupReferences":[{"link":"https://localhost/mgmt/shared/authz/resource-groups/fe7a1ebc-3e61-30aa-8a5d-c7721f7c6ce2"}]}' Please note that the reference to the user foo in the userReferences field is different from Step #1: The user created was /mgmt/tm/auth/user/foo while the reference is /mgmt/shared/authz/users/foo . The /mgmt/shared/authz/users/foo is automatically created. J ust use /mgmt/shared/authz/users/ + user name . Again, please observe that the resource-groups reference is not by name but by ID. See the selfLink in the step #3. You get the following JSON object. { "name": "testRole", "userReferences": [ { "link": "https://localhost/mgmt/shared/authz/users/foo" } ], "resourceGroupReferences": [ { "link": "https://localhost/mgmt/shared/authz/resource-groups/fe7a1ebc-3e61-30aa-8a5d-c7721f7c6ce2" } ], "generation": 1, "lastUpdateMicros": 1521682931708662, "kind": "shared:authz:roles:rolesworkerstate", "selfLink": "https://localhost/mgmt/shared/authz/roles/testRole" } Test The following REST calls demonstrate that the user "foo" can GET the virtual "vs" but nothing else. curl -D - -sku foo:foo https://192.168.226.155/mgmt/tm/ltm/virtual/vs | grep HTTP HTTP/1.1 200 OK curl -D - -sku foo:foo https://192.168.226.155/mgmt/tm/ltm/virtual | grep HTTP HTTP/1.1 401 F5 Authorization Required curl -D - -sku foo:foo https://192.168.226.155/mgmt/tm/ltm/virtual -X PATCH -d '{"test":"test"}' | grep HTTP HTTP/1.1 401 F5 Authorization Required curl -D - -sku foo:foo https://192.168.226.155/mgmt/tm/sys/version | grep HTTP HTTP/1.1 401 F5 Authorization Required Cleanup Removing users that are no longer used is a good admin practice. The cleanup procedure is describedbelow (response JSON blobs are not shown): 1. Delete the resource-group. Use ID. curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/resource-groups/fe7a1ebc-3e61-30aa-8a5d-c7721f7c6ce2 -X DELETE 2. Delete the custom role. curl -sku admin:<pass> https://<mgmtIP>/mgmt/shared/authz/roles/testRole -X DELETE 3. Delete the user. You can perform this step from tmsh or Configuration Utility. curl -sku admin:<pass> https://<mgmtIP>/mgmt/tm/auth/user/foo -X DELETEAS3 Best Practice
Introduction AS3 is a declarative API that uses JSON key-value pairs to describe a BIG-IP configuration. From virtual IP to virtual server, to the members, pools, and nodes required, AS3 provides a simple, readable format in which to describe a configuration. Once you've got the configuration, all that's needed is to POST it to the BIG-IP, where the AS3 extension will happily accept it and execute the commands necessary to turn it into a fully functional, deployed BIG-IP configuration. If you are new to AS3, start reading the following references: Products - Automation and orchestration toolchain(f5.com; Product information) Application Services 3 Extension Documentation(clouddocs; API documentation and guides) F5 Application Services 3 Extension(AS3) (GitHub; Source repository) This article describes some considerations in order to efficiently deploy the AS3 configurations. Architecture In the TMOS space, the services that AS3 provides are processed by a daemon named 'restnoded'. It relies on the existing BIG-IP framework for deploying declarations. The framework consists of httpd, restjavad and icrd_child as depicted below (the numbers in parenthesis are listening TCP port numbers). These processes are also used by other services. For example, restjavad is a gateway for all the iControl REST requests, and is used by a number of services on BIG-IP and BIG-IQ. When an interaction between any of the processes fails, AS3 operation fails. The failures stem from lack of resources, timeouts, data exceeding predefined thresholds, resource contention among the services, and more. In order to complete AS3 operations successfully, it is advised to follow the Best Practice outlined below. Best Practice Your single source of truth is your declaration Refrain from overwriting the AS3-deployed BIG-IP configurations by the other means such as TMSH, GUI or iControl REST calls. Since you started to use the AS3 declarative model, the source of truth for your device's configurations is in your declaration, not the BIG-IP configuration files. Although AS3 tries to weigh BIG-IP locally stored configurations as much as it can do, discrepancy between the declaration and the current configuration on BIG-IP may cause the AS3 to perform less efficiently or error unexpectedly. When you wish to change a section of a tenant (e.g., pool name change), modify the declaration and submit it. Keep the number of applications in one tenant to a minimum AS3 processes each tenant separately.Having too many applications (virtual servers) in a single tenant (partition) results in a lengthy poll when determining the current configuration. In extreme cases (thousands of virtuals), the action may time out. When you want to deploy a thousand or more applications on a single device, consider chunking the work for AS3 by spreading the applications across multiple tenants (say, 100 applications per tenant). AS3 tenant access behavior behaves as BIG-IP partition behavior.A non-Common partition virtual cannot gain access to another partition's pool, and in the same way, an AS3 application does not have access to a pool or profile in another tenant.In order to share configuration across tenants, AS3 allows configuration of the "Shared" application within the "Common" tenant.AS3 avoids race conditions while configuring /Common/Shared by processing additions first and deletions last, as shown below.This dual process may cause some additional delay in declaration handling. Overwrite rather than patching (POSTing is a more efficient practice than PATCHing) AS3 is a stateless machine and is idempotent. It polls BIG-IP for its full configuration, performs a current-vs-desired state comparison, and generates an optimal set of REST calls to fill the differences.When the initial state of BIG-IP is blank, the poll time is negligible.This is why initial configuration with AS3 is often quicker than subsequent changes, especially when the tenant contains a large number of applications. AS3 provides the means to partially modify using PATCH (seeAS3 API Methods Details), but do not expect PATCH changes to be performant.AS3 processes each PATCH by (1) performing a GET to obtain the last declaration, (2) patching that declaration, and (3) POSTing the entire declaration to itself.A PATCH of one pool member is therefore slower than a POST of your entire tenant configuration.If you decide to use PATCH,make sure that the tenant configuration is a manageable size. Note: Using PATCH to make a surgical change is convenient, but using PATCH over POST breaks the declarative model. Your declaration should be your single source of truth.If you include PATCH, the source of truth becomes "POST this file, then apply one or more PATCH declarations." Get the latest version AS3 is evolving rapidly with new features that customers have been wishing for along with fixes for known issues. Visitthe AS3 section of the F5 Networks Github.Issuessection shows what features and fixes have been incorporated. For BIG-IQ, check K54909607: BIG-IQ Centralized Management compatibility with F5 Application Services 3 Extension and F5 Declarative Onboarding for compatibilities with BIG-IQ versions before installation. Use administrator Use a user with the administrator role when you submit your declaration to a target BIG-IP device. Your may find your role insufficient to manipulate BIG-IP objects that are included in your declaration. Even one authorized item will cause the entire operation to fail and role back. See the following articles for more on BIG-IP user and role. Manual Chapter : User Roles (12.x) Manual Chapter : User Roles (13.x) Manual Chapter : User Roles (14.x) Prerequisites and Requirements(clouddocs AS3 document) Use Basic Authentication for a large declaration You can choose either Basic Authentication (HTTP Authorization header) or Token-Based Authentication (F5 proprietary X-F5-Auth-Token) for accessing BIG-IP. While the Basic Authentication can be used any time, a token obtained for the Token-Based Authentication expires after 1,200 seconds (20 minutes). While AS3 does re-request a new token upon expiry, it requires time to perform the operation, which may cause AS3 to slow down. Also, the number of tokens for a user is limited to 100 (since 13.1), hence if you happen to have other iControl REST players (such as BIG-IQ or your custom iControl REST scripts) using the Token-Based Authentication for the same user, AS3 may not be able to obtain the next token, and your request will fail. See the following articles for more on the Token-Based Authentication. Demystifying iControl REST Part 6: Token-Based Authentication(DevCentral article). iControl REST Authentication Token Management(DevCentral article) Authentication and Authorization(clouddocs AS3 document) Choose the best window for deployment AS3 (restnoded daemon) is a Control Plane process. It competes against other Control Plane processes such as monpd and iRules LX (node.js) for CPU/memory resources. AS3 uses the iControl REST framework for manipulating the BIG-IP resources. This implies that its operation is impacted by any processes that use httpd (e.g., GUI), restjavad, icrd_child and mcpd. If you have resource-hungry processes that run periodically (e.g., avrd), you may want to run your AS3 declaration during some other time window. See the following K articles for alist of processes K89999342 BIG-IP Daemons (12.x) K05645522BIG-IP Daemons (v13.x) K67197865BIG-IP Daemons (v14.x) K14020: BIG-IP ASM daemons (11.x - 15.x) K14462: Overview of BIG-IP AAM daemons (11.x - 15.x) Workarounds If you experience issues such as timeout on restjavad, it is possible that your AS3 operation had resource issues. After reviewing the Best Practice above but still unable to alleviate the problem, you may be able to temporarily fix it by applying the following tactics. Increase the restjavad memory allocation The memory size of restjavad can be increased by the following tmsh sys db commands tmsh modify sys db provision.extramb value <value> tmsh modify sys db restjavad.useextramb value true The provision.extramb db key changes the maximum Java heap memory to (192 + <value> * 8 / 10) MB. The default value is 0. After changing the memory size, you need to restart restjavad. tmsh restart sys service restjavad See the following article for more on the memory allocation: K26427018: Overview of Management provisioning Increase a number of icrd_child processes restjavad spawns a number of icrd_child processes depending on the load. The maximum number of icrd_child processes can be configured from /etc/icrd.conf. Please consult F5 Support for details. See the following article for more on the icrd_child process verbosity: K96840770: Configuring the log verbosity for iControl REST API related to icrd_child Decrease the verbosity levels of restjavad and icrd_child Writing log messages to the file system is not exactly free of charge. Writing unnecessarily large amount of messages to files would increase the I/O wait, hence results in slowness of processes. If you have changed the verbosity levels of restjavad and/or icrd_child, consider rolling back the default levels. See the following article for methods to change verbosity level: K15436: Configuring the verbosity for restjavad logs on the BIG-IP systemDeclarative Advanced WAF policy lifecycle in a CI/CD pipeline
The purpose of this article is to show the configuration used to deploy a declarative Advanced WAF policy to a BIG-IP and automatically configure it to protect an API workload by consuming an OpenAPI file describing the application. For this experiment, a Gitlab CI/CD pipeline was used to deploy an API workload to Kubernetes, configure a declarative Adv. WAF policy to a BIG-IP device and tuning it by incorporating learning suggestions exported from the BIG-IP. Lastly, the F5 WAF tester tool was used to determine and improve the defensive posture of the Adv. WAF policy. Deploying the declarative Advanced WAF policy through a CI/CD pipeline To deploy the Adv. WAF policy, the Gitlab CI/CD pipeline is calling an Ansible playbook that will in turn deploy an AS3 application referencing the Adv.WAF policy from a separate JSON file. This allows the application definition and WAF policy to be managed by 2 different groups, for example NetOps and SecOps, supporting separation of duties. The following Ansible playbook was used; --- - hosts: bigip connection: local gather_facts: false vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxx" tasks: - name: Deploy AS3 API AWAF policy uri: url: "https://{{ bigip }}/mgmt/shared/appsvcs/declare" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxx body: "{{ lookup('file','as3_waf_openapi.json') }}" body_format: json validate_certs: no status_code: 200 The Advanced WAF policy 'as3_waf_openapi.json' was specified as follows: { "class": "AS3", "action": "deploy", "persist": true, "declaration": { "class": "ADC", "schemaVersion": "3.2.0", "id": "Prod_API_AS3", "API-Prod": { "class": "Tenant", "defaultRouteDomain": 0, "arcadia": { "class": "Application", "template": "generic", "VS_API": { "class": "Service_HTTPS", "remark": "Accepts HTTPS/TLS connections on port 443", "virtualAddresses": ["xxxxx"], "redirect80": false, "pool": "pool_NGINX_API", "policyWAF": { "use": "Arcadia_WAF_API_policy" }, "securityLogProfiles": [{ "bigip": "/Common/Log all requests" }], "profileTCP": { "egress": "wan", "ingress": { "use": "TCP_Profile" } }, "profileHTTP": { "use": "custom_http_profile" }, "serverTLS": { "bigip": "/Common/arcadia_client_ssl" } }, "Arcadia_WAF_API_policy": { "class": "WAF_Policy", "url": "http://xxxx/root/awaf_openapi/-/raw/master/WAF/ansible/bigip/policy-api.json", "ignoreChanges": true }, "pool_NGINX_API": { "class": "Pool", "monitors": ["http"], "members": [{ "servicePort": 8080, "serverAddresses": ["xxxx"] }] }, "custom_http_profile": { "class": "HTTP_Profile", "xForwardedFor": true }, "TCP_Profile": { "class": "TCP_Profile", "idleTimeout": 60 } } } } } The AS3 declaration will provision a separate Administrative Partition ('API-Prod') containing a Virtual Server ('VS_API'), an Adv. WAF policy ('Arcadia_WAF_API_policy') and a pool ('pool_NGINX_API'). The Adv.WAF policy being referenced ('policy-api.json') is stored in the same Gitlab repository but can be downloaded from a separate location. { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ ] } The declarative Adv.WAF policy is referencing in turn the OpenAPI file ('openapi3-arcadia.yaml') that describes the application being protected. Executing the Ansible playbook results in the AS3 application being deployed, along with the Adv.WAF policy that is automatically configured according to the OpenAPI file. Handling learning suggestions in a CI/CD pipeline The next step in the CI/CD pipeline used for this experiment was to send legitimate traffic using the API and collect the learning suggestions generated by the Adv.WAF policy, which will allow a simple way to customize the WAF policy further for the specific application being protected. The following Ansible playbook was used to retrieve the learning suggestions: --- - hosts: bigip connection: local gather_facts: true vars: my_admin: "xxxx" my_password: "xxxx" bigip: "xxxxx" tasks: - name: Get all Policy_key/IDs for WAF policies uri: url: 'https://{{ bigip }}/mgmt/tm/asm/policies?$select=name,id' method: GET headers: "Authorization": "Basic xxxxxxxxxxx" validate_certs: no status_code: 200 return_content: yes register: waf_policies - name: Extract Policy_key/ID of Arcadia_WAF_API_policy set_fact: Arcadia_WAF_API_policy_ID="{{ item.id }}" loop: "{{ (waf_policies.content|from_json)['items'] }}" when: item.name == "Arcadia_WAF_API_policy" - name: Export learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: POST headers: "Content-Type": "application/json" "Authorization": "Basic xxxxxxxxxxx" body: "{ \"inline\": \"true\", \"policyReference\": { \"link\": \"https://{{ bigip }}/mgmt/tm/asm/policies/{{ Arcadia_WAF_API_policy_ID }}/\" } }" body_format: json validate_certs: no status_code: - 200 - 201 - 202 - name: Get learning suggestions uri: url: "https://{{ bigip }}/mgmt/tm/asm/tasks/export-suggestions" method: GET headers: "Authorization": "Basic xxxxxxxxx" validate_certs: no status_code: 200 register: result - name: Print learning suggestions debug: var=result A sample learning suggestions output is shown below: "json": { "items": [ { "endTime": "xxxxxxxxxxxxx", "id": "ZQDaRVecGeqHwAW1LDzZTQ", "inline": true, "kind": "tm:asm:tasks:export-suggestions:export-suggestions-taskstate", "lastUpdateMicros": 1599953296000000.0, "result": { "suggestions": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "Check maximum number of parameters" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable HTTP Check", "entity": { "description": "No Host header in HTTP/1.1 request" }, "entityChanges": { "enabled": true }, "entityType": "http-protocol" }, { "action": "add-or-update", "description": "Enable enforcement of policy violation", "entity": { "name": "VIOL_REQUEST_MAX_LENGTH" }, "entityChanges": { "alarm": true, "block": true }, "entityType": "violation" } Incorporating the learning suggestions in the Adv.WAF policy can be done by simple copy&pasting the self-contained learning suggestions blocks into the "modifications" list of the Adv.WAF policy: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "transparent", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } Enhancing Advanced WAF policy posture by using the F5 WAF tester The F5 WAF tester is a tool that generates known attacks and checks the response of the WAF policy. For example, running the F5 WAF tester against a policy that has a "transparent" enforcement mode will cause the tests to fail as the attacks will not be blocked. The F5 WAF tester can suggest possible enhancement of the policy, in this case the change of the enforcement mode. An abbreviated sample output of the F5 WAF Tester: ................................................................ "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": false, "reason": "ASM Policy is not in blocking mode", "support_id": "" } }, "system": "All systems" } }, "summary": { "fail": 48, "pass": 0 } Changing the enforcement mode from "transparent" to "blocking" can easily be done by editing the same Adv. WAF policy file: { "policy": { "name": "policy-api-arcadia", "description": "Arcadia API", "template": { "name": "POLICY_TEMPLATE_API_SECURITY" }, "enforcementMode": "blocking", "server-technologies": [ { "serverTechnologyName": "MySQL" }, { "serverTechnologyName": "Unix/Linux" }, { "serverTechnologyName": "MongoDB" } ], "signature-settings": { "signatureStaging": false }, "policy-builder": { "learnOnlyFromNonBotTraffic": false }, "open-api-files": [ { "link": "http://xxxxx/root/awaf_openapi/-/raw/master/App/openapi3-arcadia.yaml" } ] }, "modifications": [ { "action": "add-or-update", "description": "Enable Evasion Technique", "entity": { "description": "Directory traversals" }, "entityChanges": { "enabled": true }, "entityType": "evasion" } ] } A successful run will will be achieved when all the attacks will be blocked. ......................................... "100000023": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt (AWS Metadata Server)", "results": { "parameter": { "expected_result": { "type": "signature", "value": "200018040" }, "pass": true, "reason": "", "support_id": "17540898289451273964" } }, "system": "All systems" }, "100000024": { "CVE": "", "attack_type": "Server Side Request Forgery", "name": "SSRF attempt - Local network IP range 10.x.x.x", "results": { "request": { "expected_result": { "type": "signature", "value": "200020201" }, "pass": true, "reason": "", "support_id": "17540898289451274344" } }, "system": "All systems" } }, "summary": { "fail": 0, "pass": 48 } Conclusion By adding the Advanced WAF policy into a CI/CD pipeline, the WAF policy can be integrated in the lifecycle of the application it is protecting, allowing for continuous testing and improvement of the security posture before it is deployed to production. The flexible model of AS3 and declarative Advanced WAF allows the separation of roles and responsibilities between NetOps and SecOps, while providing an easy way for tuning the policy to the specifics of the application being protected. Links UDF lab environment link. Short instructional video link.F5 & Cisco ACI Essentials - Take advantage of Policy Based Redirect
Different applications and environments have unique needs on how traffic is to be handled. Some applications due to the nature of their functionality or maybe due to a business need do require that the application server(s) are able to view the real IP of the client making the request to the application. Now when the request comes to the BIG-IP it has the option to change the real IP of the request or to keep it intact. In order to keep it intact the setting on the F5 BIG-IP ‘Source Address Translation’ is set to ‘None’. Now as simple as it may sound to just toggle a setting on the BIG-IP, a change of this setting causes significant change in traffic flow behavior. Let’s take an example with some actual values. Starting with a simple setup of a standalone BIG-IP with one interface on the BIG-IP for all traffic (one-arm) Client – 10.168.56.30 BIG-IP Virtual IP – 10.168.57.11 BIG-IP Self IP – 10.168.57.10 Server – 192.168.56.30 Scenario 1: With SNAT From Client : Src: 10.168.56.30 Dest: 10.168.57.11 From BIG-IP to Server: Src: 10.168.57.10 (Self-IP) Dest: 192.168.56.30 With this the server will respond back to 10.168.57.10 and BIG-IP will take care of forwarding the traffic back to the client. Here the application server see’s the IP 10.168.57.10 and not the client IP Scenario 2: No SNAT From Client : Src: 10.168.56.30 Dest: 10.168.57.11 From BIG-IP to Server: Src: 10.168.56.30 Dest: 192.168.56.30 With this the server will respond back to 10.168.56.30 and here where comes in the complication, the return traffic needs to go back to the BIG-IP and not the real client. One way to achieve this is to set the default GW of the server to the Self-IP of the BIG-IP and then the server will send the return traffic to the BIG-IP. BUT what if the server default gateway is not to be changed for whatsoever reason. It is at this time Policy based redirect will help. The default gw of the server will point to the ACI fabric, the ACI fabric will be able to intercept the traffic and send it over to the BIG-IP. With this the advantage of using PBR is two-fold The server(s) default gateway does not need to point to BIG-IP but can point to the ACI fabric The real client IP is preserved for the entire traffic flow Avoid server originated traffic to hit BIG-IP, resulting BIG-IP to configure a forwarding virtual to handle that traffic.If server originated traffic volume is high it could result unnecessary load the BIG-IP Before we get to the deeper into the topic of PRB below are a few links to help you refresh on some of the Cisco ACI and BIG-IP concepts ACI fundamentals: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/1-x/aci-fundamentals/b_ACI-Fundamentals.html SNAT and Automap: https://support.f5.com/csp/article/K7820 BIG-IP modes of deployment: https://support.f5.com/csp/article/K96122456#link_02_01 Now let’s look at what it takes to configure PBR using a Standalone BIG-IP Virtual Edition in One-Arm mode Network diagram for reference: To use the PBR feature on APIC - Service graph is a MUST Details on L4-L7 service graph on APIC To get hands on experience on deploying a service graph (without pbr) Configuration on APIC 1) Bridge domain ‘F5-BD’ Under Tenant->Networking->Bridge domains->’F5-BD’->Policy IP Data plane learning - Disabled 2) L4-L7 Policy-Based Redirect Under Tenant->Policies->Protocol->L4-L7 Policy based redirect, create a new one Name: ‘bigip-pbr-policy’ L3 destinations: BIG-IP Self-IP and MAC IP: 10.168.57.10 MAC: Find the MAC of interface the above Self-IP is assigned from logging into the BIG-IP (example: 00:50:56:AC:D2:81) 3) Logical Device Cluster- Under Tenant->Services->L4-L7, create a logical device Managed – unchecked Name: ‘pbr-demo-bigip-ve` Service Type: ADC Device Type: Virtual (in this example) VMM domain (choose the appropriate VMM domain) Devices: Add the BIG-IP VM from the dropdown and assign it an interface Name: ‘1_1’, VNIC: ‘Network Adaptor 2’ Cluster interfaces Name: consumer, Concrete interface Device1/[1_1] Name: provider, Concrete interface: Device1/[1_1] 4) Service graph template Under Tenant->Services->L4-L7->Service graph templates, create a service graph template Give the graph a name:’ pbr-demo-sgt’ and then drag and drop the logical device cluster (pbr-demo-bigip-ve) to create the service graph ADC: one-arm Route redirect: true 5) Click on the service graph created and then go to the Policy tab, make sure the Connections for the connectors C1 and C2 and set as follows: Connector C1 Direct connect – False (Not mandatory to set to 'True' because PBR is not enabled on consumer connector for the consumer to VIP traffic) Adjacency type – L3 Connector C2 Direct connect - True Adjacency type - L3 6) Apply the service graph template Right click on the service graph and apply the service graph Choose the appropriate consumer End point group (‘App’) provider End point group (‘Web’) and provide a name for the new contract For the connector select the following: BD: ‘F5-BD’ L3 destination – checked Redirect policy – ‘bigip-pbr-policy’ Cluster interface – ‘provider’ Once the service graph is deployed, it is in applied state and the network path between the consumer, BIG-IP and provider has been successfully setup on the APIC. 7) Verify the connector configuration for PBR. Go to Device selection policy under Tenant->Services-L4-L7. Expand the menu and click on the device selection policy deployed for your service graph. For the consumer connector where PBR is not enabled Connector name - Consumer Cluster interface - 'provider' BD- ‘F5-BD’ L3 destination – checked Redirect policy – Leave blank (no selection) For the provider connector where PBR is enabled: Connector name - Provider Cluster interface - 'provider' BD - ‘F5-BD’ L3 destination – checked Redirect policy – ‘bigip-pbr-policy’ Configuration on BIG-IP 1) VLAN/Self-IP/Default route Default route – 10.168.57.1 Self-IP – 10.168.57.10 VLAN – 4094 (untagged) – for a VE the tagging is taken care by vCenter 2) Nodes/Pool/VIP VIP – 10.168.57.11 Source address translation on VIP: None 3)iRule (end of the article) that can be helpful for debugging Few differences in configuration when the BIG-IP is a Virtual edition and is setup in a High availability pair 1)BIG-IP: Set MAC Masquerade (https://support.f5.com/csp/article/K13502) 2)APIC: Logical device cluster Promiscuous mode – enabled Add both BIG-IP devices as part of the cluster 3) APIC: L4-L7 Policy-Based Redirect L3 destinations: Enter the Floating BIG-IP Self-IP and MAC masquerade ------------------------------------------------------------------------------------------------------------------------------------------------------------------ Configuration is complete, let’s take a look at the traffic flows Client-> F5 BIG-IP -> Server Server-> F5 BIG-IP -> Client In Step 2 when the traffic is returned from the client, ACI uses the Self-IP and MAC that was defined in the L4-L7 redirect policy to send traffic to the BIG-IP iRule to help with debugging on the BIG-IP when LB_SELECTED { log local0. "==================================================" log local0. "Selected server [LB::server]" log local0. "==================================================" } when HTTP_REQUEST { set LogString "[IP::client_addr] -> [IP::local_addr]" log local0. "==================================================" log local0. "REQUEST -> $LogString" log local0. "==================================================" } when SERVER_CONNECTED { log local0. "Connection from [IP::client_addr] Mapped -> [serverside {IP::local_addr}] \ -> [IP::server_addr]" } when HTTP_RESPONSE { set LogString "Server [IP::server_addr] -> [IP::local_addr]" log local0. "==================================================" log local0. "RESPONSE -> $LogString" log local0. "==================================================" } Output seen in /var/log/ltm on the BIG-IP, look at the event <SERVER_CONNECTED> Scenario 1: No SNAT -> Client IP is preserved Rule /Common/connections <HTTP_REQUEST>: Src: 10.168.56.30 -> Dest: 10.168.57.11 Rule /Common/connections <SERVER_CONNECTED>: Src: 10.168.56.30 Mapped -> 10.168.56.30 -> Dest: 192.168.56.30 Rule /Common/connections <HTTP_RESPONSE>: Src: 192.168.56.30 -> Dest: 10.168.56.30 If you are curious of the iRule output if SNAT is enabled on the BIG-IP - Enable AutoMap on the virtual server on the BIG-IP Scenario 2: With SNAT -> Client IP not preserved Rule /Common/connections <HTTP_REQUEST>: Src: 10.168.56.30 -> Dest: 10.168.57.11 Rule /Common/connections <SERVER_CONNECTED>: Src: 10.168.56.30 Mapped -> 10.168.57.10-> Dest: 192.168.56.30 Rule /Common/connections <HTTP_RESPONSE>: Src: 192.168.56.30 -> Dest: 10.168.56.30 References: ACI PBR whitepaper: https://www.cisco.com/c/en/us/solutions/collateral/data-center-virtualization/application-centric-infrastructure/white-paper-c11-739971.html Troubleshooting guide: https://www.cisco.com/c/dam/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/troubleshooting/Cisco_TroubleshootingApplicationCentricInfrastructureSecondEdition.pdf Layer4-Layer7 services deployment guide https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/L4-L7-services/Cisco-APIC-Layer-4-to-Layer-7-Services-Deployment-Guide-401/Cisco-APIC-Layer-4-to-Layer-7-Services-Deployment-Guide-401_chapter_011.html Service graph: https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/L4-L7-services/Cisco-APIC-Layer-4-to-Layer-7-Services-Deployment-Guide-401/Cisco-APIC-Layer-4-to-Layer-7-Services-Deployment-Guide-401_chapter_0111.html https://www.cisco.com/c/en/us/td/docs/switches/datacenter/aci/apic/sw/4-x/L4-L7-services/Cisco-APIC-Layer-4-to-Layer-7-Services-Deployment-Guide-401.pdfF5 & Cisco ACI Essentials - ServiceCenter: One stop shop for IP address facts
Having to debug a network issue can be a daunting task, involving 100's of IP address's spread acrossyour entire deployment. Imagine having a tool which can help with getting all the information you want of an IP address with a click of a button. The F5 ACI ServiceCenter is one such tool designed to do exactly that and more. To refresh your knowledge on the tool: Lightboard Video Troubshooting tips User and deployment guide Telemetry streaming Let's get into the nuts and bolts of getting the entire benefit from visibility from the ServiceCenter. BIG-IP has a great component as part of its automation toolchain called 'Telemetry streaming'. The ServiceCenter takes advantage of telemetry streaming (TS) to grab traffic statistics from the BIG-IP. Integration TS with the ServiceCenter is a two step process: Step 1: Download and install the TS RPM package on the BIG-IP(no cost, no license) Step 2:Configure the TS consumer on the BIG-IP which the ServiceCenter will poll. Use a tool like POSTMAN or Curl to POST the below API to the BIG-IP. URI: https://<BIG-IP MGMT IP address>/mgmt/shared/telemetry/declare Payload: { "class":"Telemetry", "My_Poller":{ "class":"Telemetry_System_Poller", "interval":0, "actions":[ { "includeData":{ }, "locations":{ "virtualServers":{ ".*":{ } }, "pool":{ ".*":{ } } } } ] }, "My_System":{ "class":"Telemetry_System", "enable":"true", "systemPoller":[ "My_Poller" ] }, "My_Pull_Consumer":{ "class":"Telemetry_Pull_Consumer", "type":"default", "systemPoller":[ "My_Poller" ] } } IP Address facts Once logged into the ServiceCenter from the APIC controller choose the telemetry consumer to retrieve the IP address statistics. Conclusion The ServiceCenter besides the statistics will give much more information like the active connections on the BIG-IP, the physical port on which the IP is active (Virtual IP or node IP) on both the BIG-IP and the Cisco ACI, filtered logs from the BIG-IP and much more. To learn more check out the video below on the latest App enhancements and example use cases: Download the App to get started: https://dcappcenter.cisco.com/f5-aci-servicecenter.htmlAgility 2020 - you're invited!
In-person event for Agility 2020 has been cancelled. Please see the Agility Event Page for more details. (Update 2/28/2020) In an abundance of caution for our customers, partners and employees, we have made the tough decision to cancel our in-person event for Agility 2020 due to the escalating travel and safety concerns related to the global COVID-19 (Coronavirus) outbreak. While we are disappointed to miss sharing ideas and solving problems with customers and partners from around the globe in person, we believe this is the best decision for everyone's welfare. We are rapidly developing an alternative to Agility as a virtual experience in the near term to deliver valuable Lab, Break-out Session, Certification and Keynote content to our customers and partners. Check back regularly for more details on the virtual event or email F5Agility@F5.com for additional information. <Professor Farnsworth imitation>Good news, everybody!</Professor Farnsworth imitation> As you know, there was no Agility 2019. This was in part so that we could reset the time of year for the conference from August to March. Agility 2020will be held from March 16-19, 2020 at theSwan & Dolphinin Orlando, Florida. Orlando, and Disney, and putt-putt golfing... That's right, *puts on ears* we're going to Disneyworld - and you are all cordially invited to participate in labs and breakouts, meet fellow F5 users, talk with F5 and partner subject-matter experts, learn to develop and deploy applications in days instead of months, secure your apps at scale in a multi-cloud environment, and hear about our vision for the future of F5 and NGINX. Registration is now open! The DevCentral team will be busy as usual that week. We areallflying over, and will be: hosting our usual booth and giving out swag in the expo hall, hosting a walk-in Nerdery zone next to our booth, where folks can drop in to speak with one of our subject-matter experts, presenting breakout sessions, hanging out at Geekfest, connecting community, enjoying the exclusive community area at the final night party, and of course, spoiling the dev/central MVPs during the joint 2019-20 MVP Summit at Agility with special sessions and activities. If you'd like to do more than pick up all the knowledge being dropped, if you have some cool technical stories or lessons-learned to share, please stay tuned for the open call for proposals which should go live in early December - so please start getting those great breakout, lightening round, and open talk ideas ready. Hope to see you there!iControl REST + jq Cookbook - Part 3: Advanced Topics
In Part 3, some useful filters from actual customer cases are presented: i.e., replication of an existing object and data-group modification. The article also shows easy ways to summarize the iControl REST log files. In the last section, a method to access the Github API to retrieve information aboutF5 Automation Toolchain. Replicate an object with slight modifications Sometimes you want to replicate a configuration object with a slight changes to a few properties. jq can do that for you. Just call a GET request against the source object, pipe the response to a jq filter to modify the values of properties, and POST the result back to the BIG-IP. In the example that follows, I will create a virtual server object from the existing virtual "vs" with the name, destination and pool properties changed. Let's check the existing properties first. curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \ jq -r '.name, .destination, .pool' vs /Common/172.16.10.50:80 /Common/Pool-CentOS80 Use the = assignment operator to modify the name property value, say to "vs8080". jq outputs the entire JSON text with only that property value changed. curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \ jq '.name="vs8080"' { "kind": "tm:ltm:virtual:virtualstate", "name": "vs8080", # name changed. "fullPath": "vs", ... By repeatedly piping the result to the next assignment, you can modify other properties too: i.e., curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \ jq '.name="vs8080" | .destination="/Common/172.16.10.50:8080" | .pool="/Common/Pool-CentO8080"' { "kind": "tm:ltm:virtual:virtualstate", "name": "vs8080", # name changed ... "destination": "/Common/172.16.10.50:8080", # destination changed ... "pool": "/Common/Pool-CentO8080", # pool changed ... Redirect the output to a file (e.g., virtual8080.json ). Post it to the BIG-IP to create the new virtual server object. curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual \ -H "Content-type: application/json" -X POST \ -d@virtual8080.json The virtual8080.json file contains some references to the original configurations: For example, the selfLink property is still pointing to the virtual vs, and the poolReferences to the Pool-CentOS80. No worry. iControl REST will relink the paths to the new objects. Optionally, you can selectively remove the properties using the del function. Specify the property names you would like to get rid of in the argument. Use the comma ( , ) for multiple properties. curl -sku admin:admin https://<host>/mgmt/tm/ltm/virtual/vs | \ jq '.name="vs8080" | .destination="/Common/172.16.10.50:8080" | .pool="/Common/Pool-CentO8080" | del(.selfLink, .poolReference, .policiesReference, .profilesReference)' Note that you do not add\to the end of second line. This part is not a shell. Modify internal data-group records One of the frequently asked questions is an iControl REST method to partially add/modify/delete data-group records. Unfortunately, there is none. You need to obtain the current record set, modify it locally, then submit a PATCH request. jq can make this process easier. You can get the records property value from the following jq filter. As you can see, the call returns an array. curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/TestDg | \ jq '.records' [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "large" } ] You can add a record object (consisting of the name and data properties) to the array using the + operator. Note that the right-hand operand must also be an array. curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \ jq '.records + [{"name": "price", "data": 7}]' [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "large" }, { "name": "price", "data": 7 } ] The patch data you want to send is an object that consists of the records property with the name and value: i.e., "records":[ {...}, {...}, {...}] . You can create the object by specifying the property name, colon ( : ) and the value and enclosing them with the {} . curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/TestDg | \ jq '{"records": (.records + [{"name": "price", "data": 7}])}' { "records": [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "large" }, { "name": "price", "data": 7 } ] } Note that the parentheses around the value are required to keep the value in one blob. Now, pipe the output to a file (here tmpData ) and PATCH it to the endpoint. curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg \ -X PATCH -H "Content-type: application/json" -d@tmpData | jq . { "kind": "tm:ltm:data-group:internal:internalstate", "name": "TestDg", "fullPath": "TestDg", "generation": 77, "selfLink": "https://localhost/mgmt/tm/ltm/data-group/internal/TestDg?ver=15.1.2", "type": "string", "records": [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "price", "data": "7" }, { "name": "type", "data": "large" } ] } Use the select function to remove a record by name. The function selectively yields the objects if the statement is the argument returns true. To exclude an object, use != . In the following example, the record with the name "price" is removed. curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \ jq '{"records": [(.records[] | select(.name != "price"))]}' { "records": [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "large" } ] } Now, pipe the output to a file and PATCH it. curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg \ -X PATCH -H "Content-type: application/json" \ -d@tmpData | jq . { "kind": "tm:ltm:data-group:internal:internalstate", "name": "TestDg", "fullPath": "TestDg", "generation": 79, "selfLink": "https://localhost/mgmt/tm/ltm/data-group/internal/TestDg?ver=15.1.2", "type": "string", "records": [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "large" } ] } Similarly, you can replace a record using the if-then-else-end syntax. The if statement accepts standard conditional operators such as == . Note that jq's if-then-else-end always requires the else block. You need to explicitly put . in the else block to output the existing value. Also, it requires end at the end of statement. The following example modifies the object's data value to "medium" if the name property value is "type": $ curl -sku $PASS https://$HOST/mgmt/tm/ltm/data-group/internal/TestDg | \ jq '{"records": [(.records[] | if .name == "type" then .data = "medium" else . end)]}' { "records": [ { "name": "name", "data": "Cha Siu Bao" }, { "name": "type", "data": "medium" } ] } Process restjavad-autit.log /var/log/restjavad-audit.*.log (where * is a number) keeps records of REST calls as shown in the example below (manually edited for readability): [I][784][12 May 2020 05:17:34 UTC][ForwarderPassThroughWorker] {"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/available", "status":400,"from":"com.f5.rest.common.RestAvailabilityEntry"} Successful GETs (200) are not recorded by default. To include these calls, change the value of the audit.logging.FileHandler.level property in /etc/restjavad.log.conf to FINE . The log messages are handy for gathering the iControl REST statistics: How often do the calls fail? Which endpoint is used most extensively? Such intelligence is helpful when you need to investigate performance issues or intermittent failure. First, use grep to extract only the audit records as the file may contain other messages (such as boot markers). Then, apply cut to trim the first timestamp part. Now you have a sequence of JSON texts. fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6- {"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/tmm-stat","status":400,"from":"192.168.184.10"} {"user":"local/admin","method":"GET","uri":"http://localhost:8100/mgmt/tm/sys/disk/stat","status":400,"from":"192.168.184.10"} {"user":"local/admin","method":"POST","uri":"http://localhost:8100/mgmt/tm/util/bash","status":200,"from":"192.168.184.10"} ... With the help of sort and uniq, you can generate a statistical report on the methods and status: fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6- | \ jq -r '[.method, (.status|tostring)] | join(" ")' | \ sort | uniq -c 3 DELETE 200 2160 GET 400 20 GET 401 14 GET 404 1 GET 500 1 GET 503 21 PATCH 200 12 PATCH 400 5 PATCH 403 147 POST 200 2 POST 202 46 POST 400 14 POST 401 4 POST 403 4 POST 404 2 PUT 202 7 PUT 400 The first part of the filter extracts the method and status properties. The status value is a number, hence it must be converted to string using the tostring function (needs the parentheses to group the two operations). Note that the join function only accepts an array with string elements. Similarly, you can obtain an endpoint-based summary. In the following example, long URLs are summarized into a component level (e.g., endpoints with actual virtual server object names are reduced to just /mgmt/tm/ltm/virutal ) by picking only the first four path components (mgmt, tm, ltm and virtual). fgrep '"user":' restjavad-audit.0.log | cut -d ' ' -f6- | \ jq -r '.uri | split("/")[3:7] | join("/")' | \ sort | uniq -c 1 mgmt/cm/autodeploy/software-image-downloads 64 mgmt/shared/authn/login 2 mgmt/shared/authz/roles 14 mgmt/shared/authz/tokens 9 mgmt/shared/authz/users ... The first part of the filter extracts the .uri property. The values are paths in string. The split function in the next stage splits the string by / character and the following slice ( [3:7] ) extracts only the first four path elements. Why starting from 3? Because the URIs start from https://localhost:8010/... and the first meaningful path element (mgmt) starts from the 3rd (counting from 0). Process restjavad-api-usage.log Another source of iControl REST statistics is /var/log/restjavad-api-usage.log . The file keeps the access counts per endpoint and per method in JSON format since the initial deployment of the system. The restjavad-api-usage feature was introduced in BIG-IP v13.1. The property apiUsageStartTimeMicrosUtc tells you when it started to tally the accesses. The value is a Unix epoch in microseconds. cat restjavad-api-usage.json | \ jq -r '.apiUsageStartTimeMicrosUtc | ./(1000*1000) | todate' 2019-08-25T22:05:46Z The usage property keeps the access counts in an array. Each array element is an object that contains the usage per endpoint. Access counts are categorized by the access sources: external : Accesses from outside (via management interface). internal : Accesses from inside the box (via localhost) messages : iControl REST internal message passing. The above access source properties keep the counts for each access method in an array: i.e., GET, POST, PUT, DELETE, PATCH and OPTIONS. Unlike the audit log, it counts the accesses irrespective of success or failure. The example below shows the 1000th element, which, in this case, contains the access counts for the /mgmt/cm endpoint. cat restjavad-api-usage.json | jq -r '.usage[1000]' # 1000th element { "path": "/mgmt/cm", # The endpoint "external": [ # Access from outside 1, # GET 0, # POST 0, # PUT 0, # DELETE 0, # PATCH 0 # OPTIONS ], "internal": [ # Access from inside (localhost) 0, 0, 0, 0, 0, 0 ], "message": [ # Interthread communications 0, 0, 0, 0, 0, 0 ] } The example below extracts the number of GET requests against /mgmt/cm from external clients: cat restjavad-api-usage.json | jq -r '.usage[1000].external[0]' 1 jq can do more for you. The example below extracts the sum of the external access counts (for all the methods) for the endpoints that start with /mgmt/tm/sys . cat restjavad-api-usage.json | \ jq -r '.usage[] | select(.path | startswith("/mgmt/tm/sys")) | [.path, (.external | add | tostring)] | join("\t")' /mgmt/tm/sys 4 /mgmt/tm/sys/clock 1 /mgmt/tm/sys/cpu 1 /mgmt/tm/sys/crypto 1 /mgmt/tm/sys/crypto/cert-order-manager 2 /mgmt/tm/sys/crypto/csr 1 /mgmt/tm/sys/crypto/key 6 /mgmt/tm/sys/db 2930 /mgmt/tm/sys/disk 5 /mgmt/tm/sys/disk/application-volume 1 /mgmt/tm/sys/disk/directory 9 /mgmt/tm/sys/file/external-monitor 7 /mgmt/tm/sys/global-settings 35 /mgmt/tm/sys/ha-group 2 /mgmt/tm/sys/hardware 4 /mgmt/tm/sys/host-info 1 /mgmt/tm/sys/provision 5 /mgmt/tm/sys/ready 16 /mgmt/tm/sys/software 4 /mgmt/tm/sys/software/image 1 /mgmt/tm/sys/tmm-info 1 /mgmt/tm/sys/ucs 4 /mgmt/tm/sys/version 94 Let's explore this filter. The select function extracts the usage object with the endpoints that start with /mgmt/tm/sys . .path | startswith() in the select argument feeds the .path value to the startswith function, which returns true when the input starts with the string in the function argument. The .external part of (.external | add | tostring) filter extracts the array with 6 numbers. The add function adds them together, then the last tostring does the number to string conversion (because join requires string elements). Get a list of the latest Automation Toolchain F5 Automation Toolchain is a collection of remote deployment services (SeeF5 Automation Toolchain). The software, which you need to install on BIG-IP to utilize the service(s), is updated with enhancements and bug fixes fairly often, hence you may want to check Github from time to time to stay up to date (The Github pages areAS3,DOandTSrespectively). You can automate the checking by accessing the following URI. AS3 (Application Services 3): https://api.github.com/repos/F5Networks/f5-appsvcs-extension/releases/latest DO (Declarative Onboarding): https://api.github.com/repos/F5Networks/f5-declarative-onboarding/releases/latest TS (Telemetry Streaming): https://api.github.com/repos/F5Networks/f5-telemetry-streaming/releases/latest A GET request to the URL returns a JSON formatted text that contains information about the latest software. Here is an example from AS3. curl -sk https://api.github.com/repos/F5Networks/f5-appsvcs-extension/releases/latest | \ jq -r '.' { ... "node_id": "MDc6UmVsZWFzZTM0MjA3MjM4", "tag_name": "v3.24.0", "target_commitish": "master", "name": "v3.24.0", "draft": false, "prerelease": false, "created_at": "2020-11-20T02:03:49Z", "published_at": "2020-11-20T02:12:39Z", ... } The top-level properties show the generic information about the software package. The version number is shown in the name property and the date of publication is found in the published_at property. Here's a script to grab these information for all the services: for service in f5-appsvcs-extension f5-declarative-onboarding f5-telemetry-streaming; do echo -n "$service ..." curl -sk https://api.github.com/repos/F5Networks/$service/releases/latest | \ jq -r '[.name, .published_at] | join(" ")' done f5-appsvcs-extension ...v3.24.0 2020-11-20T02:12:39Z f5-declarative-onboarding ...v1.17.0 2020-11-20T01:45:51Z f5-telemetry-streaming ...v1.16.0 2020-11-20T01:22:22Z Conclusion iControl REST was introduced in BIG-IP version 11.4 and it is now a must-know feature for automated configurations and maintenance. It represents the information in JSON, hence jq is a natural choice for casual data processing. If you did not find the method you need in this series, please take a look at thejq manual. You should be able to find the functions you are after. Having said so, jq is not as flexible as a general-purpose programming language. If you have complex requirements, you should consider using iControl REST SDKs such asF5 Python SDK.iControl REST + jq Cookbook - Part 2: Intermediate
Part 2 deals with slightly complex use cases: Manipulating array elements, looking for strings that match a certain criteria and handling both property names (keys) and values to generate a tabularized output. iControl REST sometimes returns a bare output from the underlying tmsh (e.g., monitor stats). Toward the end, filters for such row strings are discussed. Additionally, methods for converting Unix epoch timestamps are presented. If you find some jq notations difficult, please revisit thePart 1. Extract an array element A single array may contain a large number of elements. For example, the records property of an LTM internal data-group (e.g., tmsh list ltm data-group internal aol ) stores as many entries as you want in a single array. Here's an example from the AOL internal data-group: curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | jq '.records' [ { "name": "64.12.96.0/19", "data": "" }, ... { "name": "207.200.112.0/21", "data": "" } ] You can selectively extract a single element by specifying the array index number to the records property. The index number starts from 0, so [1] means the 2nd element. curl -sku $PASS https://<host>/mgmt/tm/ltm/data-group/internal/aol | jq '.records[1]' { # 0th record element "name": "195.93.16.0/20", "data": "" } You can specify multiple indexes separated by , . In the example below, only the name properties are extracted by additional | .name . curl -sku $PASS https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[2, 3, 5] | .name' 195.93.48.0/22 # The name property value from 2nd 195.93.64.0/19 # The name property value from 3rd 198.81.0.0/22 # The name property value from 5th Note that jq does not raise any error for an out-of-range index: It silently returns null (just like referencing a non-existing property). curl -sku $PASS https://<host>/mgmt/tm/ltm/data-group/internal/aol | jq '.records[100]' null Slice an array You can extract a subset of array by slicing: Just specify the start (inclusive) and stop (exclusive) indexes separated by colon ( : ) inside the [] . For example, to extract the names (IP addresses) from the top 4 elements in the AOL data-group records, run the following command: curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[0:4] | .[].name' 64.12.96.0/19 # The name property value from 0th 195.93.16.0/20 195.93.48.0/22 195.93.64.0/19 # The name property value from 3rd You might have noticed the subtle difference from the previous comma separated indexes: The additional [] is prepended to the .name filter. This is because the comma separated indexing ( .records[2, 3, 5] ) yields three independent objects (with the name and data properties), while the slice generates an array containing the record objects. Because it is an array, you need to iterate through it to process each record. When the start index number is omitted, zero is used ( [0:4] and [:4] are identical). When the end index number is omitted, it assumes the end of the array. The following example extracts the last 4 IP addresses from the AOL data-group (it currently contains 14 records) - from the 10th to the 13th. curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[10:] | .[].name' 205.188.146.144/30 # The name property value from 10th 205.188.192.0/20 205.188.208.0/23 207.200.112.0/21 # The name property value from last Applying the slice to a string yields its substring. For example, you can extract the first 8 characters from "iControlREST" like this: $ echo '"iControlREST"' | jq '.[:8]' # Piping "iControlREST" to jq "iControl" Find the element Having said that, we seldom look for elements by their index numbers. We normally look for a pattern. For example, you want to find the data value of the "useragent6" in the sys_APM_MS_Office_OFBA_DG internal data-group. curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/sys_APM_MS_Office_OFBA_DG | \ jq -r '.records[] | [.name, .data] | join("\t")' ie_sp_session_sharing_enabled 0 ie_sp_session_sharing_inactivity_timeout 60 ofba_auth_dialog_size 800x600 useragent1 microsoft data access internet publishing provider useragent2 office protocol discovery useragent3 microsoft office useragent4 non-browser useragent5 msoffice 12 useragent6 microsoft-webdav-miniredir # Looking for this one useragent7 webdav-miniredir useragent9 ms frontpage 1[23456789] useragent10 onenote You can extract only the specific element using the select function. Specify the condition for the selection in the function argument. Usual conditional statements that yields a Boolean can be used: For example, the == operator returns true if two operands are equal. echo '["BIG-IP", "BIG-IQ"]' | jq '.[] | . == "BIG-IP"' true # BIG-IP is BIG-IP false # BIG-IQ is not BIG-IP The select function outputs the input only when the condition in the argument is true. echo '["BIG-IP", "BIG-IQ"]' | jq '.[] | select(. == "BIG-IP")' "BIG-IP" With these techniques, you can extract the record (object) that contains the "useragent6" in the name property: curl -sku $PASS https://<host>/mgmt/tm/ltm/data-group/internal/sys_APM_MS_Office_OFBA_DG | \ jq -r '.records[] | select(.name == "useragent6")' { "name": "useragent6", "data": "microsoft-webdav-miniredir" } The following comparison operators can be used in jq: != , > , >= , < and <= . Find matching elements The select function accepts anything as the conditional statement as long as it returns true or false . For example, the startswith and endswith functions. They compare the input against their argument and return a Boolean. Unlike == , where you explicitly compare the incoming data ( . ) and the string, you only need to specify the string to compare. Let's find the IP addresses that start with "195" in the AOL internal data-group. curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[] | .name | select(startswith("195"))' 195.93.16.0/20 195.93.48.0/22 195.93.64.0/19 195.93.96.0/19 Here is an example for endswith , looking for the "/20" subnets: curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[].name | select(endswith("/20"))' 195.93.16.0/20 198.81.16.0/20 205.188.112.0/20 205.188.192.0/20 Regular expressions? Yes, jq comes with standard regular expression functions such as test and match . Here's an example for finding IP addresses that contain 96, dot and any digits: curl -sku admin:admin https://<host>/mgmt/tm/ltm/data-group/internal/aol | \ jq -r '.records[] | .name | select(test("96.\\d+"))' 64.12.96.0/19 195.93.96.0/19 The jq regular expression is Perl-compatible, however note that backslashes must be escaped (e.g., \\d+ instead of just \d+ ). For more information on the regular expression feature, refer tojq manual. The RegEx functions are available only when the jq was compiled with the regular expression libraryOniguruma. Without the library, you will receive the jq: error (at <stdin>:0): jq was compiled without ONIGURUMA regex libary. match/test/sub and related functions are not available error upon the use of any RegEx function. The jq on a BIG-IP doesnotcome with the library. Extract property names and values Sometimes, you need to obtain both property names (keys) and their values, however, the .[] filter only yields the values. To get both, filter the incoming objects with the to_entries function (it acts like the Python's items() ). For example, GET /mgmt/tm/ltm/monitor/http/http ( tmsh list ltm monitor http http equivalent) returns the following object: curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http | jq '.' { "kind": "tm:ltm:monitor:http:httpstate", "name": "http", ... } The to_entries generates an array. Its elements are objects and each object consists of two properties: "key" and "value". The key property keeps the original property name while the value property stores the value of the property. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http | jq 'to_entries' [ { # Object from the 0th property "key": "kind", # 0th property name "value": "tm:ltm:monitor:http:httpstate" # 0th property value }, { "key": "name", "value": "http" }, ... ] By piping the result to .key, .value , you now have both property names and values. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http | \ jq -r 'to_entries[] | .key, .value' kind # 0th property name tm:ltm:monitor:http:httpstate # 0th property value name # 1st property name http # 1st property value ... Obviously, it's much better to have the key and value in the same line. You can do so by using the array constructor [] and the join function as described inPart 1. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http | \ jq -r 'to_entries[] | [.key, .value] | join(" ... ")' kind ... tm:ltm:monitor:http:httpstate name ... http fullPath ... http generation ... 0 selfLink ... https://localhost/mgmt/tm/ltm/monitor/http/http?ver=15.1.2 adaptive ... disabled adaptiveDivergenceType ... relative ... Extract a property from a deeply nested object Some iControl REST responses are deeply nested. For example, the BIG-IP version number is found in the 6th level in a GET /mgmt/tm/sys/version response: curl -sku admin:admin https://<host>/mgmt/tm/sys/version | jq '.' { "kind": "tm:sys:version:versionstats", "selfLink": "https://localhost/mgmt/tm/sys/version?ver=15.1.1", "entries": { # top level "https://localhost/mgmt/tm/sys/version/0": { # 2nd level "nestedStats": { # 3rd level "entries": { # 4th level "Build": { # 5th level "description": "0.0.6" # 6th level }, ... "Version": { "description": "15.1.1" # HERE! } ... Obviously, you can reference the description property in the Version object by chaining the property names with dots from top to bottom (you need to double-quote the URL-like property as it contains sensitive characters). curl -sku admin:admin https://<host>/mgmt/tm/sys/version | \ jq '.entries."https://localhost/mgmt/tm/sys/version/0".nestedStats.entries.Version.description' "15.1.1" That's too tedious (and error prone). In this case, you can use [] as a catch-all property for the 2nd, 3rd and 4th levels. As you have seen, the iterator [] yields multiple elements in an array or object, however, because these levels only contain one property, it is as good as referencing specifically. curl -sku admin:admin https://<host>/mgmt/tm/sys/version | \ jq '.entries[][][].Version.description' "15.1.1" Tabularize all the version information Producing a tabularized version information should be easy by now: Combination of the -r option and the to_entries and join functions that you have already seen. curl -sku admin:admin https://<host>/mgmt/tm/sys/version | \ jq -r '.entries[][][] | to_entries[] | [.key, .value.description] | join("\t")' Build 0.0.6 Date Thu Oct 8 02:52:59 PDT 2020 Edition Point Release 0 Product BIG-IP Title Main Package Version 15.1.1 The .entries[][][] yields the 5th level objects: e.g., "Build": {"description": "0.0.6"} . The to_entries function then generates an object with the key and value properties: The key is "Build" and the "value" is {"description":"0.0.6"} in this case. You can access the data by referencing .key and .value.description . Process raw monitor output Some iControl REST calls return the raw output from tmsh. For example, GET /mgmt/tm/ltm/monitor/http/http/stats returns a blob of lines from tmsh show ltm monitor http http in the .apiRawValues.apianonymous property. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http/stats | \ jq '.apiRawValues.apiAnonymous' "---------------------------\nLTM::Monitor /Common/http \n------------------ ..." The output is a single string with the line feeds expressed in the literal "\n" (backslash + n). You can simply output it using the --raw-output option. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http/stats | \ jq -r '.apiRawValues.apiAnonymous' --------------------------- LTM::Monitor /Common/http --------------------------- Destination: 10.200.20.10:80 State time: up for 8hrs:9mins:13sec | Last error: N/A @2021.01.06 08:44:24 ... Hmmm ... This is too easy for the intermediate level. Let's achieve the same goal in a harder way: Replace the \n (2F 6E) with LF (0A) using the string replacement (substitution) function sub . sub requires the regular expression library. curl -sku admin:admin https://<host>/mgmt/tm/ltm/monitor/http/http/stats | \ jq -r '.apiRawValues.apiAnonymous | sub("\\n"; "\n")' --------------------------- LTM::Monitor /Common/http --------------------------- Destination: 10.200.20.10:80 State time: down for 6hrs:3mins:51sec | Last error: No successful responses received before deadline. @2020.10.30 10:49:33 ... The first argument of the sub function is afromstring and the second one is atostring. The argument separator is ; (semicolon). Note that you need to escape the literal backslash by another backslash ( \\n ). You can also use regular expressions. Format 'list sys global-settings' output Another response that needs massaging might be the one from tmsh list sys global-settings file-blacklist-path-prefix . As you can see below, it returns a single string, consisting of paths that are enclosed in {} . tmsh list sys global-settings file-blacklist-path-prefix sys global-settings { file-blacklist-path-prefix "{/shared/3dns/} {/shared/bin/} {/shared/core/} ... } Naturally, the equivalent iControl REST call returns a very long string. We can check how long it is by passing the string to the length function. It returns a number of characters when a string is fed (a number of elements or properties in an array or object): curl -sku admin:admin https://$HOST/mgmt/tm/sys/global-settings | \ jq '.fileBlacklistPathPrefix | length' 469 # 469 characters long You can split the string at each whitespace using the split function (this is not a regular expression function). The function generates an array consisting of substrings. curl -sku admin:admin https://<host>/mgmt/tm/sys/global-settings | \ jq -r '.fileBlacklistPathPrefix | split(" ")' [ "{/shared/3dns/}", "{/shared/bin/}", "{/shared/core/}", ... ] If you think the curly braces are obnoxious, you can remove them by feeding it to the gsub regular expression function: a sibling of sub that performs substitution globally. gsub requires the regular expression library. curl -sku admin:admin https://<host>/mgmt/tm/sys/global-settings| \ jq -r '.fileBlacklistPathPrefix | split(" ") | .[] | gsub("{|}"; "")' /shared/3dns/ /shared/bin/ /shared/core/ ... The first argument for gsub is afromstring. Here, a regular expression ("matches either { or } ") is used. The second argument is atostring. Here, any { or } is globally replaced with an empty string. Again, note that the argument separator is ; . You can use the sub command for global replacement by specifying the g (global) flag in the third argument. curl -sku admin:admin https://<host>/mgmt/tm/sys/global-settings | \ jq -r '.fileBlacklistPathPrefix | split(" ") | .[] | sub("{|}"; ""; "g")' Convert Unix epoch time Some time related property values are expressed in Unix epoch in microseconds (10 -6 ). For example, the lastUpdateMicros and expirationMicros properties in an authentication token object represent the timestamps for last update and expiration in microseconds. curl -sku admin:admin https://<host>/mgmt/shared/authz/tokens/LKMRTZ3TZY5PFHJI5HZNRCDUEM | \ jq -r '.lastUpdateMicros, .expirationMicros' 1604038137666701 # Last Update date/time in μs 1604039337667000 # Expiration date/time in μs You can convert the numbers to human readable format using the todate function. curl -sku admin:admin https://<host>/mgmt/shared/authz/tokens/LKMRTZ3TZY5PFHJI5HZNRCDUEM | \ jq -r '.lastUpdateMicros, .expirationMicros | ./(1000*1000) | todate' 2020-10-30T06:08:57Z 2020-10-30T06:28:57Z The expression in the middle of the filter is an arithmetic computation: It divides the incoming time ( . , which is in microsecond) by 10 6 . This is required because the todate function expects the Unix epoch time in seconds. The literal 1000*1000 looks ugly, but it is better than 1000000 (IMHO). If you prefer, you can use the pow function to compute 10 to the power of negative 6. $ echo 1604039337667000 | jq '. * pow(10; -6) | todate' "2020-10-30T06:28:57Z" jq is equipped with standard C math functions (such as cos and log ). Refer to thejq manual. Continue toPart 3.
