Tinkering with the BIGREST Python SDK - Part 2

A couple months back, Leonardo Souza (one of our MVPs) released a new SDK for interacting with the iControl REST interface. A write up right here on DevCentral covers the high level basics of why he rolled his own when we already had one (f5-common-python). In part one I covered typical CRUD operations. In this article, I'm going to compare and contrast operational functionality of the old SDK with Leo's new SDK, adding commentary after each section and concluding with overall impressions.

Uploading and Downloading Files

We’ll start with file management since we’ll need some files in place for a couple of the following sections. The upload/download options are shown below. For things like (small) config files, certs, keys, iFiles, etc, I upload everything to /var/config/rest/downloads. Images and big files always to /shared/images. For non-UCS downloads, I move files to /shared/images, download the appropriate files, then delete them as necessary.

Description          Method  URI                                             File Location
Upload Image         POST    /mgmt/cm/autodeploy/sotfware-image-uploads/*    /shared/images
Upload File          POST    /mgmt/shared/file-transfer/uploads/*            /var/config/rest/downloads
Upload UCS           POST    /mgmt/shared/file-transfer/ucs-uploads/*        /var/local/ucs
Download UCS         GET     /mgmt/shared/file-transfer/ucs-downloads/*      /var/local/ucs
Download Image/File  GET     /mgmt/cm/autodeploy/sotfware-image-downloads/*  /shared/images

For this example, I'll upload the myfile.txt file that contains a pool configuration that I'll merge later in this article, and I'll download an image file.

f5-common-python

# Upload myfile.txt to /var/config/rest/downloads/
fcp.shared.file_transfer.uploads.upload_file('myfile.txt')
# Download BIGIP-13.1.3-0.0.6.iso from /shared/images/
fcp.cm.autodeploy.software_image_downloads.download_file('BIGIP-13.1.3-0.0.6.iso', 'BIGIP-13.1.3-0.0.6.iso')

bigrest

# Upload myfile.txt to /var/config/rest/downloads/
br.upload('/mgmt/shared/file-transfer/uploads/', 'myfile.txt')
# Download BIGIP-13.1.3-0.0.6.iso from /shared/images/
br.download('/mgmt/cm/autodeploy/sotfware-image-downloads/', 'BIGIP-13.1.3-0.0.6.iso')

Conclusion

Given that the iControlREST interface itself is not very helpful in keeping track of where to upload/download files, it’s hardly a fault of either SDK in approach. That said, I like bigrest’s angle in that the endpoints are direct tmsh mappings, so as long as you have the table memorized or available, you won’t have to hunt in the SDK source to find out how to manage files, so that’s a big win. If I were to make a tweak to bigrest here, however, I’d leave the upload/download methods as is but add a type (image/ucs/file) argument so the SDK would handle the correct endpoint for me. The tradeoff there though is violating the consistency of bigrest in forcing the user to supply the endpoints, and isn’t future-proofed in the event those endpoints change. So maybe I’m walking myself out of that argument… :)

Running Commands

Sometimes instead of managing the BIG-IP configuration, you need to interact with the system, whether it’s loading or saving a configuration or checking dns resolution from the system’s perspective. For this illustration, I’ll show how to merge the following myfile.txt configuration file that I uploaded to my BIG-IP with the above steps.

# File to merge: myfile.txt
ltm pool mergepool {
    members {
        192.168.103.20:http {
            address 192.168.103.20
            session monitor-enabled
            state down
        }
    }
    monitor http
}

f5-common-python

fcp.tm.ltm.pools.pool.exists(name='mergepool')
False
options = {}
options['file'] = '/var/config/rest/downloads/myfile.txt'
options['merge'] = True
fcp.tm.sys.config.exec_cmd('load', options=[options])
fcp.tm.ltm.pools.pool.exists(name='mergepool')
True

bigrest

br.exist('/mgmt/tm/ltm/pool/mergepool')
False
options = {}
options['file'] = '/var/config/rest/downloads/myfile.txt'
options['merge'] = True
data = {}
data['command'] = 'load'
data['options'] = [options]
br.command('/mgmt/tm/sys/config', data)
br.exist('/mgmt/tm/ltm/pool/mergepool')
True

Conclusion

In both cases I have to pack some data to supply the tmsh sys load command. F5-common-python does some of the work for me, but bigrest requires me to pack both blobs. But again, the simple command method is more clear with bigrest than f5-common-python’s exec_cmd.

Working with Stats

The stats interface in iControl REST is not for the faint of heart. Both libraries provide assistance to make the return data more immediately usable.

f5-common-python

from f5.utils.responses.handlers import Stats
pool = fcp.tm.ltm.pools.pool.load(name='testpool')
poolstats = Stats(pool.stats.load())
for k, v in poolstats.stat.items():
    if v.get('description') != None:
        print(f'{k}: {v.get("description")}')
    elif v.get('value') != None:
        print(f'{k}: {v.get("value")}')

# Result
activeMemberCnt: 0
availableMemberCnt: 0
...
status_availabilityState: offline
status_enabledState: enabled
status_statusReason: The children pool member(s) are down
totRequests: 0

bigrest

poolstats = br.show('/mgmt/tm/ltm/pool')
for p in poolstats:
    for k, v in p.properties.items():
        if v.get('description') != None:
            print(f'{k}: {v.get("description")}')
        elif v.get('value') != None:
            print(f'{k}: {v.get("value")}')

# Result
...same as above from f5-common-python

Conclusion

Stats management is built-in with bigrest with no requirement to load additional utilities, and it is bound to the show method which makes a lot of sense since that’s how they are consumed on the tmsh cli. They are stored slightly differently in the sdk objects, with f5-common-python storing them as a list of dictionaries and bigrest storing them as a dictionary of dictionaries, but once that is managed on the python side, the data is the data.

Working with Request Parameters

You can read more about using request parameters with the iControl REST interface here and here. For this example, I’ll pull the BIG-IP’s list of pools, but only their name and load balancing mode attributes by utilizing the select parameter.

f5-common-python

data = {}
data['params'] = '$select=name,loadBalancingMode'
pools = fcp.tm.ltm.pools.get_collection(requests_params=data)
for p in pools:
...     print(p)
...
{'name': 'mergepool', 'loadBalancingMode': 'round-robin'}
{'name': 'nerdlife_pool', 'loadBalancingMode': 'round-robin'}
{'name': 'testpool', 'loadBalancingMode': 'round-robin'}
{'name': 'p1', 'loadBalancingMode': 'round-robin'}

bigrest

pools = br.load('/mgmt/tm/ltm/pool/?$select=name,loadBalancingMode')
for p in pools:
...     print(p)
...
{
    "name": "mergepool",
    "loadBalancingMode": "round-robin"
}
{
    "name": "nerdlife_pool",
    "loadBalancingMode": "round-robin"
}
{
    "name": "testpool",
    "loadBalancingMode": "round-robin"
}

Conclusion

Because the endpoint URIs are specified directly in bigrest, that makes using the request parameters much more simple. In f5-common-python, you have to package the parameters and pass it as an object to the method, which is a little clunky.

Working with Transactions

Sometimes changes are dependent on other changes, for example, updating key and certificate files while they are already applied to a clientssl profile in use. Or, you create some helper objects while preparing to create a virtual server, but the virtual server creation fails and leaves you with some artifacts. In cases like that, you will want to use a transaction so that any of the pre-work that is accomplished before an error doesn’t actually get committed to the system. For this example, I’ll create a pool and then try to create an already existing virtual server to see the failure, then create a virtual server to show success.

f5-common-python

# Import the transaction manager
from f5.bigip.contexts import TransactionContextManager
# Setup the transaction
tx = fcp.tm.transactions.transaction
# Execute the transaction
with TransactionContextManager(tx) as api:
    api.tm.ltm.pools.pool.create(name='testpool_fcp')
    api.tm.ltm.virtuals.virtual.create(name='testvip', pool='testpool_fcp', destination='192.168.102.199:80')
# Error because the virtual already exists
f5.sdk_exception.TransactionSubmitException: 409 Unexpected Error: Conflict for uri: https://ltm3.test.local:443/mgmt/tm/transaction/1599859075279313/
Text: '{"code":409,"message":"transaction failed:01020066:3: The requested Virtual Server (/Common/testvip) already exists in partition Common.","errorStack":[],"apiError":2}'
# Note that the pool was not created
fcp.tm.ltm.pools.pool.exists(name='testpool_fcp')
False

# Execute the transaction
with TransactionContextManager(tx) as api:
    api.tm.ltm.pools.pool.create(name='testpool_fcp')
    api.tm.ltm.virtuals.virtual.create(name='testvip199', pool='testpool_fcp', destination='192.168.102.199:80')
# No error, check to see if pool and virtual exist
fcp.tm.ltm.pools.pool.exists(name='testpool_fcp')
True
fcp.tm.ltm.virtuals.virtual.exists(name='testvip199')
True

bigrest

# Execute the transaction
with br as transaction:
    data = {}
    data['name'] = 'testpool_br'
    br.create('/mgmt/tm/ltm/pool', data)
    data = {}
    data['name'] = 'testvip'
    data['pool'] = 'testpool_br'
    data['destination'] = '192.168.102.199:80'
    br.create('/mgmt/tm/ltm/virtual', data)
raise RESTAPIError(response, self.debug)
# Error because the virtual already exists
bigrest.common.exceptions.RESTAPIError:
Status:
409
Response Body:
{
    "code": 409,
    "message": "transaction failed:01020066:3: The requested Virtual Server (/Common/testvip) already exists in partition Common.",
    "errorStack": [],
    "apiError": 2
}
# Note that the pool was not created
br.exist('/mgmt/tm/ltm/pool/testpool_br')
False

# Execute the transaction
with br as transaction:
    data = {}
    data['name'] = 'testpool_br'
    br.create('/mgmt/tm/ltm/pool', data)
    data = {}
    data['name'] = 'testvip199'
    data['pool'] = 'testpool_br'
    data['destination'] = '192.168.102.199:80'
    br.create('/mgmt/tm/ltm/virtual', data)
# No error, check to see if pool and virtual exist
br.exist('/mgmt/tm/ltm/pool/testpool_br')
True
br.exist('/mgmt/tm/ltm/virtual/testvip199')
True

Conclusion

There’s a lot less setup required with bigrest transactions, so on that front alone I think it’s easier to work with. But bigrest has another ace up it’s sleeve by also supporting specific transaction methods: transaction_create, transaction_validate, and transaction_commit. Here’s an example of their use, and how you can interrogate the commands in a transaction should you need to remove or reorder any.

from bigrest.bigip import BIGIP

device = BIGIP('ltm3.test.local', 'admin', 'admin')

transaction_create = device.transaction_create()
transaction_id = transaction_create.properties["transId"]
print(f"Transaction ID: {transaction_id}.")
pdata = {}
pdata["name"] = 'pool_name3'
device.create("/mgmt/tm/ltm/pool", pdata)
vdata = {}
vdata["name"] = 'virtual_name3'
vdata['pool'] = 'pool_name3'
vdata["destination"] = "10.17.0.3%0:80"
device.create("/mgmt/tm/ltm/virtual", vdata)
tx1 = device.load(f"/mgmt/tm/transaction/{transaction_id}/commands/1")[0]
tx2 = device.load(f"/mgmt/tm/transaction/{transaction_id}/commands/2")[0]
print(tx1, tx2)
device.transaction_validate()
device.transaction_commit()
print(f"Transaction {transaction_id} completed.")

#output
Transaction ID: 1599862284314421.
{
    "method": "POST",
    "uri": "https://localhost/mgmt/tm/ltm/pool",
    "body": {
        "name": "pool_name3"
    },
    "evalOrder": 1,
    "commandId": 1,
    "kind": "tm:transaction:commandsstate",
    "selfLink": "https://localhost/mgmt/tm/transaction/1599862284314421/commands/1?ver=13.1.1.5"
} {
    "method": "POST",
    "uri": "https://localhost/mgmt/tm/ltm/virtual",
    "body": {
        "name": "virtual_name3",
        "pool": "pool_name3",
        "destination": "10.17.0.3%0:80"
    },
    "evalOrder": 2,
    "commandId": 2,
    "kind": "tm:transaction:commandsstate",
    "selfLink": "https://localhost/mgmt/tm/transaction/1599862284314421/commands/2?ver=13.1.1.5"
}
Transaction 1599862284314421 completed.

That's a pretty cool differentiator for bigrest!

Working with Tasks

In iControl REST, a task is an asynchronous operation that allows you to kick off a long-lived process like performing a UCS backup or running a cli script. Not every endpoint has an associated task available, so you’ll need to consult the user guides for the complete list for your version.

f5-common-python

This SDK does not have task support.

bigrest

# Import datetime to show task creation/completion times
from datetime import datetime
data = {}
data["command"] = "save"
task = br.task_start("/mgmt/tm/task/sys/config", data)
print(f'Task {task.properties["_taskId"]} created at {datetime.now().strftime("%H:%M:%S")}')
br.task_wait(task)
if br.task_completed(task):
	br.task_result(task)
	print(f'Task {task.properties["_taskId"]} completed at {datetime.now().strftime("%H:%M:%S")}')
else:
	raise Exception()

# Result
Task 1599880917597158 created at 22:21:57
Task 1599880917597158 completed at 22:22:07

Conclusion

Obviously this is a default win for bigrest since there is no support at all for tasks in f5-common-python, but it’s also super easy.

Debugging

When facing issues with what is and is not working, it’s helpful to see what the SDK is actually sending to the BIG-IP.

f5-common-python

fcp = ManagementRoot('ltm3.test.local', 'admin', 'admin', debug=True)
fcp.debug
True
pool_list = fcp.tm.ltm.pools.get_collection()
fcp.debug_output
["curl -k -X GET https://ltm3.test.local:443/mgmt/tm/ltm/pool/ -H 'User-Agent: python-requests/2.23.0 f5-icontrol-rest-python/1.3.13' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' -H 'Content-Type: application/json' -H 'Cookie: BIGIPAuthCookie=51E86ABC11D59EC24E393DBD608921A4EB9678C9; BIGIPAuthUsernameCookie=admin' -H 'Authorization: Basic YWRtaW46YWRtaW4='"]
fcp = ManagementRoot('ltm3.test.local', 'admin', 'admin', debug=True, token=True)
pool_list = fcp.tm.ltm.pools.get_collection()
fcp.debug_output
["curl -k -X GET https://ltm3.test.local:443/mgmt/tm/ltm/pool/ -H 'User-Agent: python-requests/2.23.0 f5-icontrol-rest-python/1.3.13' -H 'Accept-Encoding: gzip, deflate' -H 'Accept: */*' -H 'Connection: keep-alive' -H 'Content-Type: application/json' -H 'X-F5-Auth-Token: 4KB7K2XUPBPRF3P2GO37A5ZA4D'"]

bigrest

br = BIGIP('ltm3.test.local', 'admin', 'admin', debug='br_output.txt'

Conclusion

For the life of me I could not figure out why the debug option in bigrest was not working, until I had a failure with an API call! There are a few differences here. One, f5-common-python stores output in the instantiated object's debug_output attribute (fcp.debug_output in our case,) whereas bigrest uses files for the output. Two, bigrest only writes to the debug file when there is a failure. Three, bigrest logs actual request/response information, whereas f5-common-python output's only what the request would look like in a curl command. This is useful with out-of-band verification of the functionality, but lacks the actual details. I see the value in both approaches and would like to see both as options.

Closing Thoughts

Whew! We made it through most of the BIG-IP functionality of both libraries. What we didn't cover at all is the BIG-IQ functionality, which bigrest supports much more fully than f5-common-python, which implemented a few endpoints but that's about it.
The examples in these intro articles are simple discovery exercises, so my likes and dislikes might not hold when transitioning into real coding assignments. I’m going to take a look in my next article on bigrest at refactoring some existing f5-common-python scripts into bigrest-based scripts, so stay tuned for that!

Before my good friend Tim Rupp left F5, we had considered a major overhaul of f5-common-python with a lot of design choices that Leo made with bigrest. But now that that this excellent work has been completed in bigrest, and considering that F5’s strategy going forward will be primarily focused on the declarative model, it hardly makes sense for us to continue to invest development effort in f5-common-python.

Go therefore and install bigrest, mess around with it, and post your thoughts in the comments below! Well done, Leo!