Automating Spotfire® from scripting tools

- April 2024: this article is currently under review for a desired update by the author -

Introduction

Do you ever wish you could run a particular set of Spotfire^® tasks automatically? At IQVIA we often see the need to automate rote tasks - e.g. refreshing embedded data in an analysis, deploying new versions from our test folder to our production folders, extracting SBDF data after data wrangling processes and so on. Spotfire provides a mechanism to support such automation, but it's hard to use and requires someone with c# skills to spend time working with the Spotfire SDK. This drove me to build a 'simple' wrapper for the Spotfire COM capability that implements a few capabilities and is easy to use from tools such as PowerShell.

There's a readme file in the download that gives more info, but the basic instructions are to unzip the files, create a Spotfire object, enable any logging you want, connect to the Spotfire server and then perform actions.

Using the library

First, instantiate an object using the IQVIA.Spotfire.ComClient.Spotfire class. E.g. in PowerShell you can use this code to load the library and create the object:

Add-Type -path ./IQVIA.Spotfire.ComClient.dll
$spotfire = New-Object IQVIA.Spotfire.ComClient.Spotfire

Now enable any logging you want to support (you can either not log at all, log to the console, or use log4net configuration for more complex scenarios), then connect to the Spotfire server:

$spotfire.EnableConsoleLogging("Info") 
$spotfire.ConnectToSpotfire("/server:https://your-server-url")

Next use the API to perform any activities you want (e.g. run automation services scripts, run IronPython scripts, open files - see the readme file for details):

$spotfire.RunAutomationJobFromFile("./myjobfile") 
$spotfire.RunScript("print('hello')")

Sometimes you may need to wait for Spotfire to complete it's processing (e.g. you might have opened a file that uses data functions and Spotfire will return before the data functions have completed).

 $spotfire.WaitUntilSpotfireIdle()

And finally close Spotfire:

 $spotfire.Exit()

As a simple example, consider the PowerShell code below, which connects to Spotfire and runs an automation services job script, outputing status information to the PowerShell window.:

Add-Type -Path .\IQVIA.Spotfire.ComClient.dll
$spotfire = New-Object IQVIA.Spotfire.ComClient.Spotfire
$spotfire.EnableConsoleLogging("Info")
$spotfire.ConnectToSpotfire("/server:https://ourServerURL")
$spotfire.RunAutomationJobFromFile("c:\temp\myjobfile.xml")
$spotfire.Exit()

How do I install it?

Simply download the "SpotfireAutomation.zip" or "Spotfire Automation_before_10.3.zip" file (for versions earlier than 10.3 - it may be required for versions between 7.14 and 10.3 as I haven't a server to test it on) from this page and unzip to a folder on your drive. The only prerequisite is that you need to have a working Spotfire Analyst installation.

What can it do?

There are just a few simple capabilities

Logging

EnableConsoleLogging(logLevel) - set up simple console logging. Available levels are "Info", "Error", "Debug" and "Trace".
EnableFileLogging(logLevel, logFileName) - set up simple logging to a file. Available levels are "Info", "Error", "Debug" and "Trace".
ConfigureLogging(configFilename) - set up log4net logging using a configuration file.

Connecting

ConnectToSpotfire(commandLine) - Connect to Spotfire using command line parameters (see Spotfire documentation for details on the command line, but at a minimum use /server:url).
Exit() - Closes Spotfire.

Spotfire actions

OpenFileFromLibrary(libraryPath) - Opens a file from the Spotfire library.
RunAutomationJobFromFile(filename) - Run an Automation Services job loaded from a file. Returns True or False based on success.
RunScript(script) - Runs an IronPython script. Note that scripts can be run with or without loading a Spotfire file first. You can return values from your script by outputting them using the IronPython print() function.
WaitUntilSpotfireIdle() - Wait for Spotfire to become idle. We wait until there are no background processes running in the last 5 seconds - sometimes a background process might kick off another background process, so we need to wait a while after the last one finishes to be sure there is nothing to come.

Utilities

AddComView(viewType) - Use this for more complex COM automation where you provide your own 'view' that runs inside Spotfire (see Spotfire SDK documentation for details).
SilenceProgress() - This can be used to ensure that Spotfire's progress dialog form is suppressed. This can be important in situations such as running automation scripts within Docker containers or Windows Services, where attempting to show a dialog causes an exception.
GetLastExceptionAsString() - this will return any exception messages from previous API calls.

Logging limitations

There are a couple of limitations associated with logging:

Logging can only be activated within Spotfire once it completes connecting to a server, it isn't possible to collect log messages during the phase where Spotfire is opening and connecting. This impacts all types of logging (console, file or log4net configured)
It isn't possible to use log4net configuration to force Spotfire to log to the console. If you need console logging you should call the EnableConsoleLogging(logLevel) method.

What's that I see in your automation script about copying folders?

I also wrote a custom extension to Spotfire that adds various automation services tasks to help with things like deploying from test to production folders. If there's interest (and I can find time), I'll try and tidy it up and upload to another wiki page.

Why no 'save' or 'export data' methods?

Because you can easily write short IronPython scripts to perform those actions and run them via the RunScript(script) method. I chose not to implement lots of methods in the library and instead have only implemented the bare bones, or where supported APIs don't exist (e.g. running automation scripts).

What do I do if I get a confusing error when executing 'Add-Type'?

Sometimes Windows will mark the files as not trusted when they're downloaded from the Internet. In this situation the 'Add-Type' command will fail with a very confusing error message likely mentioning error code 0x801311515. If you see this, you simply need to open Windows Explorer, navigate to the folder containing the files, select all the ".dll" files, right-click to choose the properties - you should now see a message at the bottom of the window warning that the files is blocked so simply check the box to trust the files.

What do I do if I get a confusing 'COM' error when connecting to the Spotfire server?

This means that your Spotfire Analyst installation isn't correctly registered within the Windows registry. Spotfire normally registers itself during installation, but sometimes this can be broken (e.g. if you install multiple versions, then uninstall one) or never present (e.g. if you use a 'portable client'). In this situation you can use the following PowerShell to configure Windows appropriately - first set your current directory to the location of the Spotfire portable client, then execute the script:

cd <your Spotfire client folder>
$comstart = Get-ChildItem -Recurse -Force -Filter SpotfireDxpComStarter64.exe
$spotfire = Get-ChildItem -Filter Spotfire.Dxp.exe
if (!$comStart) {
    Write-Host "Error - SpotfireDxpComStarter64.exe not found. Registry entry for COM support won't be updated"
} elseif (!$spotfire) {
    Write-Host "Error - You must execute this script in the Spotfire portable client folder - where Spotfire.Dxp.exe is located"
} else {
    $comStartCommand = "`"$($comStart.FullName)`" `"$($Spotfire.FullName)`""
    [void] (New-PSDrive -PSProvider Registry -Root HKEY_CLASSES_ROOT -Name HKCR -ErrorAction SilentlyContinue)
    [void] (Remove-Item -Path "HKCR:Spotfire.Dxp" -Force -Recurse)
    [void] (New-Item -Path "HKCR:Spotfire.Dxp" -Force)
    [void] (New-Item -Path "HKCR:Spotfire.Dxp\CLSID" -value "{8B9A5DE9-DEB8-11DB-97F1-0011D8B1A5F4}" -Force)
    [void] (New-Item -Path "HKCR:CLSID\{8B9A5DE9-DEB8-11DB-97F1-0011D8B1A5F4}" -Force)
    [void] (New-Item -Path "HKCR:CLSID\{8B9A5DE9-DEB8-11DB-97F1-0011D8B1A5F4}\LocalServer32" -value "$comStartCommand" -Force)
    [void] (New-Item -Path "HKCR:CLSID\{8B9A5DE9-DEB8-11DB-97F1-0011D8B1A5F4}\ProgId" -value "Spotfire.Dxp" -Force)
    [void] (Remove-PSDrive -Name HKCR)
}

Is it supported?

Short answer - 'No'! I built this for our use and hope it might be useful for others. Feel free to pop questions in the feedback section and I'll endeavor to reply or try and fix any issues.

In fact, I'm using several 'hacks' in order to implement features like running automation scripts because the Spotfire API doesn't provide supported mechanisms - so there's every chance that things will break in different versions of Spotfire. The current version of this tool was tested with Spotfire 7.10, but should work with other versions (we've been using similar capabilities since around Spotfire 5.5).

Downloads

spotfireautomation.zip

spotfire_automation_before_10.3.zip

Sign In