Demystifying iControl REST Part 4: Working with JSON Data

iControl REST. It’s iControl SOAP’s baby, brother, introduced back in TMOS version 11.4 as an early access feature but released fully in version 11.5.

Several articles on basic usage have been written on iControl REST so the intent here isn’t basic use, but rather to demystify some of the finer details of using the API. This article will cover the details on how to work with the json data format used by iControl REST.

Formatting returned data for inspection

When first working with iControl REST, you might simply need to display the data queried with a browser REST client, or with curl from the command line. However, that doesn’t look too pretty by default:

[root@localhost:Active:Standalone] config # curl -sk -u admin:admin https://172.16.44.15/mgmt/tm/ltm/rule/linkinfo
{"kind":"tm:ltm:rule:rulestate","name":"linkinfo","fullPath":"linkinfo","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/rule/linkinfo?ver=12.0.0","apiAnonymous":"when FLOW_INIT {\n    \tlog local0. \"Lasthop ID: [LINK::lasthop id], Type: [LINK::lasthop type], Name: [LINK::lasthop name].\"\n    \tlog local0. \"Nexthop ID: [LINK::nexthop id], Type: [LINK::nexthop type], Name: [LINK::nexthop name].\"\n}"}[root@localhost:Active:Standalone]

That’s where the following formatting options come into play. There are examples for perl and php floating around as well, but I’ve yet to make them work locally on the BIG-IP, so I haven’t included them here to avoid confusion.

  • python -m json.tool (all versions)
  • json-format (12.0+)
  • jq . (confirmed in 12.0, but if not in 11.x versions, the executable can be installed and referenced accordingly)

Using each of these, results in a nice formatting of the data:

[root@localhost:Active:Standalone] config # curl -sk -u admin:admin \
https://172.16.44.15/mgmt/tm/ltm/rule/linkinfo | python -m json.tool
{
    "apiAnonymous": "when FLOW_INIT {\n    \tlog local0. \"Lasthop ID: [LINK::lasthop id], Type: [LINK::lasthop type], Name: [LINK::lasthop name].\"\n    \tlog local0. \"Nexthop ID: [LINK::nexthop id], Type: [LINK::nexthop type], Name: [LINK::nexthop name].\"\n}",
    "fullPath": "linkinfo",
    "generation": 1,
    "kind": "tm:ltm:rule:rulestate",
    "name": "linkinfo",
    "selfLink": "https://localhost/mgmt/tm/ltm/rule/linkinfo?ver=12.0.0"
}
[root@localhost:Active:Standalone] config # curl -sk -u admin:admin \
https://172.16.44.15/mgmt/tm/ltm/rule/linkinfo | json-format
{
  "kind": "tm:ltm:rule:rulestate",
  "name": "linkinfo",
  "fullPath": "linkinfo",
  "generation": 1,
  "selfLink": "https://localhost/mgmt/tm/ltm/rule/linkinfo?ver\u003d12.0.0",
  "apiAnonymous": "when FLOW_INIT {\n    \tlog local0. \"Lasthop ID: [LINK::lasthop id], Type: [LINK::lasthop type], Name: [LINK::lasthop name].\"\n    \tlog local0. \"Nexthop ID: [LINK::nexthop id], Type: [LINK::nexthop type], Name: [LINK::nexthop name].\"\n}"
}
[root@localhost:Active:Standalone] config # curl -sk -u admin:admin \
https://172.16.44.15/mgmt/tm/ltm/rule/linkinfo | jq .
{
  "kind": "tm:ltm:rule:rulestate",
  "name": "linkinfo",
  "fullPath": "linkinfo",
  "generation": 1,
  "selfLink": "https://localhost/mgmt/tm/ltm/rule/linkinfo?ver=12.0.0",
  "apiAnonymous": "when FLOW_INIT {\n    \tlog local0. \"Lasthop ID: [LINK::lasthop id], Type: [LINK::lasthop type], Name: [LINK::lasthop name].\"\n    \tlog local0. \"Nexthop ID: [LINK::nexthop id], Type: [LINK::nexthop type], Name: [LINK::nexthop name].\"\n}"
}

If you are on the BIG-IP command line, you will notice that with jq, you also get pretty printing with color, clarifying the key/value pairs. Also with jq, you can return specific fields in the data, see this article for more details. Of course, you might want to do more than just query data. You might also want to use the REST API to create or modify objects as well. You will need to create the POST/PUT/PATCH payload for this, and it should be in JSON format as well. You don’t need to fully populate an object’s fields to successfully submit and create/modify it, you just need to make sure the required fields are present. That changes for each object and for the nature of the changes you are making to an object. Follow the tmsh documentation’s lead for this, either from the reference guide or from interrogating objects in the tmsh shell. For example, to create a pool, you just need to supply the pool name.

curl -sk -u admin:admin https://172.16.44.15/mgmt/tm/ltm/pool \
  -H 'Content-Type: application/json' \
  -X POST \
  -d '{"name":"myNewPool"}'

Interrogating returned data programmatically

So browser tools and curl are great, but that’s a lot of typing if you are going to be using the REST API frequently to perform tasks. This is where a programming language of your choice comes into play. By moving all of your work into code, you are automating the tedium of typing (and remembering) all the nuances of working with the interface.

My language of choice is python. For one, it has a nice interpreter shell where you can code line by line and see results. Python’s json module makes it really easy to interrogate data returned from the BIG-IP, and to package data for presenting to the BIG-IP in an object type of choice. Let’s start the shell by typing “python” at the cli and getting the environment ready to make queries.

>>> import requests, json
>>> b = requests.session()
>>> b.auth = ('admin', 'admin')
>>> b.verify = False
>>> b.headers.update({'Content-TYpe':'application/json'})
>>> b_url_base = 'https://172.16.44.15/mgmt/tm'
>>> dg_details = ['internal', 'images']

Now we can make a query to the API, substituting the base url (https://172.16.44.15/mgmt/tm,) the data-group type (internal,) and the name (images.) The requests module object includes the payload and headers, methods, etc, from http, so we need to append another method to interrogate the payload. Let’s start with .text.

>>> dg1 = b.get('%s/ltm/data-group/%s/%s' % (b_url_base, dg_details[0], dg_details[1])).text
>>> type(dg1)
<type 'unicode'>

You can see that the .text method results in a unicode string:

u'{"kind":"tm:ltm:data-group:internal:internalstate","name":"images","fullPath":"images","generation":1,"selfLink":"https://localhost/mgmt/tm/ltm/data-group/internal/images?ver=12.0.0","type":"string","records":[{"name":".bmp"},{"name":".gif"},{"name":".jpg"}]}'

This is helpful to see what your return data is, but not very helpful programmatically. If you try to tickle the kind attribute on a string, you’ll get an error. But by switching from the .text method to .json(), it’ll convert the json data to the native python dictionary object.

>>> dg2 = b.get('%s/ltm/data-group/%s/%s' % (b_url_base, dg_details[0], dg_details[1])).json()
>>> type(dg)
<type 'dict'>
>>> dg2
{u'kind': u'tm:ltm:data-group:internal:internalstate', u'name': u'images', u'generation': 1, u'records': [{u'name': u'.bmp'}, {u'name': u'.gif'}, {u'name': u'.jpg'}], u'fullPath': u'images', u'type': u'string', u'selfLink': u'https://localhost/mgmt/tm/ltm/data-group/internal/images?ver=12.0.0'}

Now that the data is in a dictionary, you can tickle the attributes of the dg2 object. Let’s print out the records from that dictionary:

>>> dg2 = b.get('%s/ltm/data-group/%s/%s' % (b_url_base, dg_details[0], dg_details[1])).json()
>>> for record in dg2['records']:
...     print record['name']
...
.bmp
.gif
.jpg

So that’s great for return data. But what about data you need to push to the BIG-IP? Well, that’s also easy! Let’s step through this in the python shell:

>>> dg3 = {}
>>> dg3['name'] = 'images2'
>>> dg3['type'] = 'string'
>>> records = ['.png', '.bmp', '.jpg']
>>> record_obj = []
>>> for record in records:
...     nr = [ {'name': record}]
...     record_obj.extend(nr)
...
>>> dg3['records'] = record_obj
>>> dg3
{'records': [{'name': '.png'}, {'name': '.bmp'}, {'name': '.jpg'}], 'type': 'string', 'name': 'images2'}
>>> b.post('%s/ltm/data-group/internal' % b_url_base, json.dumps(dg3)).json()
{u'kind': u'tm:ltm:data-group:internal:internalstate', u'name': u'images2', u'generation': 330, u'records': [{u'name': u'.bmp'}, {u'name': u'.jpg'}, {u'name': u'.png'}], u'fullPath': u'images2', u'type': u'string', u'selfLink': u'https://localhost/mgmt/tm/ltm/data-group/internal/images2?ver=12.0.0'}

Start by creating an empty dictionary and assigning to dg3. Then populate the name and type keys. For the records, iterate through the image type list, extending the records obj to add to the dictionary. After verifying the dictionary is populated correctly, the post can be issued. Notice in that command the json.dumps(dg3)? That is where python converts the native dictionary type to json data before submitting the payload. I added the .json() on the end to verify the images2 data-group was actually created.

Conclusion

There are a lot of ways to work with json data. This article covered a couple approaches you can use in your journey to world domination. What’s your approach? Any tools on your tool belt you’d like to share with the community? Drop a comment below!

Published Nov 10, 2015
Version 1.0
devops
iControlREST
python
series-demystifying-icontrol-rest

Was this article helpful?

4 Comments

Sort By
  • Adam_Burnett_18's avatar
    Adam_Burnett_18
    Icon for Nimbostratus rankNimbostratus
    Nov 18, 2015
    One thing I find particularly mystifying about the iControl REST API is the hodgepodge of data type representations. For example the boolean allowNat on a pool is "yes"/"no" where the boolean manual-resume on a monitor is "enabled"/"disabled". The monitors array on a pool is an "and" delimited string. How can we as the community push for a more "standards" based API? Is there a doc describing the API entities and allowable values for each?
  • JRahm's avatar
    JRahm
    Icon for Admin rankAdmin
    Nov 18, 2015
    @adam, this is less an API issue and more an issue with the underlying tmsh, as the API is just a fancy wrapper for tmsh. The documentation will be in the tmsh reference guide.
  • Only1masterbla1's avatar
    Only1masterbla1
    Icon for Cirrus rankCirrus
    Oct 29, 2016

    In the above example, if I prefix datagroup name with a partition name, it does not work for POST. e,g, dg3['name'] = '~Common~images2' . I'm using lab VE 11.5.3 code. Since I'm not able to modify. I delete and recreate the datagroup which gets created in Common partition, but specifying any partition, including Common, does not work.