Getting Started with iRules LX, Part 5: Troubleshooting
When you start writing code, you will eventually run into some issue and need to troubleshoot. We have a few tools available for you.
Logging
One of the most common tools use for troubleshooting is logging. STDOUT and STDERR for the Node.js processes end up in the file
/var/log/ltm
. By temporarily inserting functions such as console.log
, console.error
, etc. you can easily check the state of different objects in your code at run time.
There is one caveat to logging in iRules LX; logging by most of most processes on BIG-IP is throttled by default. Basically, if 5 messages from a process come within 1 second, all other messages with be truncated. For Node.js, every line is a separate message so something like a stack trace will get automatically truncated. To remove this limit we can change the DB variable
log.sdmd.level
to the value debug
in TMSH as such -
root@(test-ve)(cfg-sync Standalone)(Active)(/Common)(tmos) # modify sys db log.sdmd.level value debug
SDMD is the process that manages the Node.js instances. This turns on verbose logging for SDMD and all the Node.js processes so you will want to turn this back to
info
when you are finished troubleshooting.
Once that is done you can start adding logging statements to your code. Here, I added a line to tell me when the process starts (or restarts) -
var f5 = require('f5-nodejs'); console.log('Node.js process starting.'); var ilx = new f5.ILXServer();
The logs would then show me this -
Jun 7 13:26:08 test-ve info sdmd[12187]: 018e0017:6: pid[21194] plugin[/Common/test_pl1.ext1] Node.js process starting.
Node Inspector
BIG-IP comes packaged with Node Inspector for advanced debugging. We won't cover how to use Node Inspector because most developers skilled in Node.js already know how to use it and there are already many online tutorials for those that dont.
Note: We highly recommend that you do not use Node Inspector in a production environment. Debuggers lock the execution of code at break points pausing the entire application, which does not work well for a realtime network device. Ideally debugging should only be done on a dedicated BIG-IP (such as a lab VE instance) in an isolated development environment.
Turn on Debugging
The first thing we want to do is to put our extension into single concurrency mode so that we are only working with one Node.js process while debugging. Also, while we are doing that we can put the extension into debugging mode. This can be done from TMSH like so -
root@(test-ve)(cfg-sync Standalone)(Active)(/Common)(tmos) # list ilx plugin DC_Plugin ilx plugin DC_Plugin { disk-space 156 extensions { dc_extension { } dc_extension2 { } } from-workspace DevCentralRocks staged-directory /var/ilx/workspaces/Common/DevCentralRocks } root@(test-ve)(cfg-sync Standalone)(Active)(/Common)(tmos) # modify ilx plugin DC_Plugin extensions { dc_extension { concurrency-mode single command-options add { --debug } }} root@(test-ve)(cfg-sync Standalone)(Active)(/Common)(tmos) # list ilx plugin DC_Plugin ilx plugin DC_Plugin { disk-space 156 extensions { dc_extension { command-options { --debug } concurrency-mode single } dc_extension2 { } } from-workspace DevCentralRocks staged-directory /var/ilx/workspaces/Common/DevCentralRocks } root@(test-ve)(cfg-sync Standalone)(Active)(/Common)(tmos) # restart ilx plugin DC_Plugin immediate
Increase ILX::call Timeout
Also, if you recall from part 3 of this series, the TCL command
ILX::call
has a 3 second timeout by default that if there is no response to the RPC, the command throws an exception. We don't need to worry about this for the command ILX::notify
because it does not expect a response from the RPC. While we have the debugger paused on a break point you are pretty much guaranteed to hit this timeout, so we need to increase it to allow for our whole method to finish execution. To do this, you will need to add arguments to the ILX::call
highlighted in red -
ILX::call $ilx_handle -timeout big_number_in_msec method arg1
This number should be big enough to walk through the entire method call while debugging. If it will take you 2 minutes, then the value should be 120000. Once you update this make sure to reload the workspace.
Launching Node Inspector
Once you have put Node.js into debugging mode and increased the timeout, you now you can fire up Node Inspector. We will need to find the debugging port that our Node.js process has been assigned via TMSH -
root@(eflores-ve3)(cfg-sync Standalone)(Active)(/Common)(tmos) # show ilx plugin DC_Plugin <----------------snipped -----------------> --------------------------------- | Extension Process: dc_extension --------------------------------- | Status running | PID 25975 | TMM 0 | CPU Utilization (%) 0 | Debug Port 1033 | Memory (bytes) | Total Virtual Size 1.2G | Resident Set Size 5.8K
Now that we have the debugging port we can put that and the IP of our management interface into the Node Inspector arguments -
[root@test-ve:Active:Standalone] config # /usr/lib/node_modules/.bin/node-inspector --debug-port 1033 --web-host 192.168.0.245 --no-inject Node Inspector v0.8.1 Visit http://192.168.0.245:8080/debug?port=1033 to start debugging.
We simply need to take the URL given to us and put it in our web browser. Then we can start debugging away!
Conclusion
We hope you have found the Getting Started with iRules LX series very informative and look forward to hearing about how iRules LX has helped you solved problems. If you need any help with using iRules LX, please dont hesitate to ask a question on DevCentral.