Using the rpdb Python Debugger with the Spotfire® Streaming EventFlow Python Operator

Introduction

At least as of Spotfire® Streaming 10.6.1, there's no special support for Python debuggers built into the Python Operator itself. Therefore, in order to figure out how to use a Python debugger with the Python opearator, we look to how to use existing python debuggers along with the Python operator.

There are any number of Python debuggers out there. One way to get a debugger to interact with Python Operator-invoked python scripts is to have the python script itself invoke a debugger that can remote attach to the python interpreter process running under StreamBase control.

Example

Here's a quick example. A lot of the python remote debugging packages require an active ssh server on the machine where the Python operator is running. It's fairly uncommon for corporate developers and deployers who use Windows to be able to easily set up ssh on Windows machines; that is, it's not usually there and it can be challenging to make it so in a controlled environment.

Therefore, one option for Windows-based developers is to use the rpdb package. This has its quirks, but seems to mostly work.

rpdb is package that allows the use of pdb-style debugging but instead presenting its user interface using the python interpreter's stdin/stdout, it presents a TCP connection-based interface that can be accessed with a TCP client such as telnet.

Here's a simple recipe, though certainly there are any number of ways to go about this kind of thing.

Install rpdb into your python environment

(python374) C:\Users\sbarber>pip install rpdb
Collecting rpdb
  Downloading rpdb-0.1.6.tar.gz (4.3 kB)
Building wheels for collected packages: rpdb
  Building wheel for rpdb (setup.py) ... done
  Created wheel for rpdb: filename=rpdb-0.1.6-py3-none-any.whl size=3156 sha256=b9e20e647fdcaf2d3b411ae122386126d47a012126b2950804cb949c8f548350
  Stored in directory: c:\users\sbarber\appdata\local\pip\cache\wheels\c1\30\77\7662f61c62bd6b8c498fb0d1ba7c65be451f24c631ccf29478
Successfully built rpdb
Installing collected packages: rpdb
Successfully installed rpdb-0.1.6

Import rpdb and Invoke rpdb.setTrace() from somewhere in the python script. For example, using the Streaming sample_python project's P01-version sample script, edit it like this:

import platform
import time
import rpdb

python = platform.python_implementation()
rpdb.set_trace()
version = platform.python_version()

Now run the EventFlow module that will invoke the Python operator. In this case, send a tuple into the Rerun input stream.

The processing of the input tuple will cause the Python script to be invoked and that in turn will soon executed the set_trace() method. So, once you think that the set_trace() method has been invoked, connect to rpdb's TCP port (4444 by default). I used a Telnet connection from the PuTTy client to do this, but any telnet client should work:

There's no particular reason the connection has to be from localhost; could be from a remote machine.

Here's what we see once PuTTy connects:

Not exactly being sure how rpdb is supposed to work, but assuming it should work just like regular ol' pdb, let's try to print out the value of a global variable:

Not sure what's going on here, apparently the first command to rpdb always gives a SyntaxError, no matter what we type. So, just enter any command, let it give the SyntaxError, and then do whatever pdb-ish thing you want:

Now go ahead and let the script execution complete using a 'c' command to let the script complete and then the Rerun tuple processes to completion.

At this point, the PuTTy Telnet session seems dead or stuck or something, though maybe I just don't know enough about rpdb. I find that to be able to get a debugger prompt upon the next set_trace() invocation that opening a new telnet session works. An inconvenience but not a blocker!