Introduction

The Script Management APIs were added in Spotfire version 12.0. The aim of the APIs is to enable automated management of IronPython and JavaScript scripts in a Spotfire document. This article will show why you might wish to use the APIs and how you can use them. 

Why use the APIs? Possible reasons could be to:

1. Add new IronPython or JavaScript scripts to a Spotfire document
2. Clean up old scripts within a Spotfire document
3. Upgrade existing scripts
4. Add JavaScript scripts to Spotfire Text Areas
5. Deploy and use your own instance of JQuery or other JavaScript library that your Spotfire analysis uses

One important caveat: as of the time of writing this article, it is not possible to automate the API over multiple Spotfire files in the Spotfire library. Spotfire Automation Services does not support the execution of IronPython scripts, but an extension has been written to enable this. Contact dr@spotfire.com to evaluate if the extension can be deployed to your Automation Services instance. Due to this, you'll need to open each Spotfire file in turn, paste in any IronPython code that you have developed, execute it, and save the file back to the library.

All the examples shown in this article will use IronPython to interact with the APIs.

The ScriptManager

The ScriptManager manages all the scripts in a Spotfire document. You can use it to access the scripts stored in the document and update them.

To retrieve all the scripts in a document:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

sm = Document.ScriptManager
for script in sm.GetScripts():
	print script.Name + ": " + script.Language.Language

In my case, running the script produces the following output:

 Execute Test2 DataFunction: IronPython
 Set Calculated Column Expression: IronPython

I have two IronPython scripts:

• one to execute a Data Function called "Test2"
• another to set a calculated column expression

 Each script object retrieved from the ScriptManager is a ScriptDefinition object - this defines the script itself - the language used, the script code, and more.

It's pretty easy to get the script code from the script objects:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

sm = Document.ScriptManager
for script in sm.GetScripts():
	print script.Name + ": " + script.Language.Language
	print script.ScriptCode

In my case, running this produces the following output:

Execute Test2 DataFunction: IronPython
# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

from Spotfire.Dxp.Data.DataFunctions import DataFunctionExecutorService, DataFunctionInvocation, DataFunctionInvocationBuilder

for dataFunction in Document.Data.DataFunctions:
	if dataFunction.Name == 'Test2':		
		dataFunction.Execute()

Set Calculated Column Expression: IronPython

mycalcCol = Document.Data.Tables["myDataTable"].Columns["mycalcCol"].As[CalculatedColumn]()

mycalcCol.Expression = "[ActualSales] * [0.5]"

You'll notice that these scripts have some hard-coded values in them - in the following example, I will demonstrate how to update a hard-coded value in a scripts stored in the document.

Updating a Script

The mechanism for updating a script is the same, regardless of which language the script uses. You can follow exactly the same process for JavaScript or IronPython scripts.

In the following scriptlet, I'm retrieving the "Set Calculated Column Expression" script by name, and updating the multiplication factor from 0.5 to 0.9:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.
from Spotfire.Dxp.Application.Scripting import ScriptDefinition

sm = Document.ScriptManager

isFound, oldScriptDef = Document.ScriptManager.TryGetScript("Set Calculated Column Expression")

print "Old code:"
print oldScriptDef.ScriptCode

newScriptCode = oldScriptDef.ScriptCode.replace("0.5", "0.9")

# Make a copy of the oldScriptDef, but with the new script code
newScriptDef = oldScriptDef.WithScriptCode(newScriptCode)

print "New code:"
print newScriptDef.ScriptCode

# Replace the old script in the ScriptManager
sm.Replace(oldScriptDef, newScriptDef)

While I'm at it, I can explain some interesting things going on here:

1. Getting scripts by name - the ScriptManager.TryGetScript method has an "out" parameter - the ScriptDefinition - study the line isFound, oldScriptDef = Document.ScriptManager.TryGetScript.... -  it assigns two variables in one! isFound will be a boolean that indicates if the script has been found by the ScriptManager, oldScriptDef will be an instance of a ScriptDefinition object, containing all we need to know about the script. You'll often see older IronPython scripts that use the common language runtime (clr) to create a reference to a ScriptDefinition object and pass it as an out parameter. However, this can issues further down the line and isn't recommended. It will create a StrongBox wrapper around the ScriptDefinition object - so you may see "Expected type, got StrongBox([type])" kinds of errors when using the reference later in your code
2. ScriptDefinition objects are immutable - this means that they cannot be modified after they have been created. Therefore, if you want to modify a script, you'll need to create a copy of it, changing the various parts of the ScriptDefinition along the way. To make this easier, the ScriptDefinition object has various convenience methods - WithDescription, WithLanguage, WithName, etc.. These methods will create a copy of the ScriptDefinition instance and return it to you. You can even "chain" them. For example: newScriptDefinition = oldScriptDefinition.WithDescription("New Description").WithScriptCode("import System")

Of course, this is a very simple example, but it's possible to replace the script code entirely, say with the contents of a file read from the file system. Explaining how to do this is beyond the scope of this article. Just search for examples of reading files in Python to figure out how to do that.

Creating a New Script

Creating a new script is really easy. This example shows how to create an IronPython script and a JavaScript script and add it to the document using the ScriptManager:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

from Spotfire.Dxp.Application.Scripting import ScriptDefinition, ScriptLanguage, ScriptParameterCollection

sm = Document.ScriptManager

# IronPython:
scriptName = "My new IronPython Script"
scriptDescription = "A very nice script"
scriptCode = "sm = Document.ScriptManager"
scriptLanguage = ScriptLanguage.IronPython27
# Create an empty collection of script parameters
scriptParameters = ScriptParameterCollection([])

newIronPythonScript = ScriptDefinition.Create(scriptName, scriptDescription, scriptCode, scriptLanguage, scriptParameters, True)

sm.AddScriptDefinition(newIronPythonScript)

# JavaScript:
scriptName = "My new JavaScript Script"
scriptDescription = "I love JavaScript"
scriptCode = "let hello = ""Hello World"""
scriptLanguage = ScriptLanguage.JavaScript
# Create an empty collection of script parameters
scriptParameters = ScriptParameterCollection([])

newJavaScriptScript = ScriptDefinition.Create(scriptName, scriptDescription, scriptCode, scriptLanguage, scriptParameters, True)

sm.AddScriptDefinition(newJavaScriptScript)

Perhaps the trickiest bit is creating parameters, so I'll explain that next!

Script Parameters

Most scripts have parameters - variables that are passed in as arguments when executing the script. Parameters make scripts flexible and reusable. The APIs allow parameters to be defined in a ScriptDefinition instance:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

from Spotfire.Dxp.Application.Scripting import ScriptDefinition, ScriptLanguage, ScriptParameterCollection, ScriptParameter, ScriptParameterType

sm = Document.ScriptManager

# IronPython:
scriptName = "My new IronPython Script with parameters"
scriptDescription = "A very nice script"
scriptCode = "sm = Document.ScriptManager"
scriptLanguage = ScriptLanguage.IronPython277

# Create some script parameters
scriptParameterArray = [ScriptParameter("Param1", "Description 1", ScriptParameterType.String), \
	ScriptParameter("Param2", "Description 2", ScriptParameterType.Visualization)]

scriptParameters = ScriptParameterCollection(scriptParameterArray)

newIronPythonScript = ScriptDefinition.Create(scriptName, scriptDescription, scriptCode, scriptLanguage, scriptParameters, True)

sm.AddScriptDefinition(newIronPythonScript)

There - not too scary after all!

Working with JavaScript Scripts and HTML Text Areas

JavaScript scripts can be included in Spotfire HTML Text Areas - a common use for this is to introduce some nice user interface elements. The favourite is the accordion control! However, a cautionary reminder - TIBCO does not recommend using JavaScript to modify the Spotfire HTML Document Object Model (DOM). Doing so is indeed possible, but can be a problem when it comes to maintenance. Future versions of Spotfire will more than likely have a different structure for the DOM. Any custom JavaScript that modifies that DOM may well fail.

The script management APIs introduced methods for working with JavaScript Scripts in text areas. These are managed through the HtmlTextAreaScriptCollection class. The collection of scripts assigned to an HTML Text Area will be included in the HTML rendering of the Text Area in Spotfire.

To demonstrate, this scriptlet will iterate over the scripts associated with a particular Text Area:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

from Spotfire.Dxp.Application.Visuals import HtmlTextArea

# Cast the Visualization object as an HtmlTextArea object
textArea = textArea.As[HtmlTextArea]()

# Iterate over the scripts used in this HtmlTextArea
for script in textArea.Scripts:
	print script.ScriptDefinition.Name

In my case, running the script produces the following output:

 My new JavaScript Script Accordion

Iterating over the scripts in an HtmlTextArea differs slightly from iterating over scripts using the ScriptManager. The returned script objects are of the type HtmlTextAreaScript, hence why, to get the name of the script being used, it's necessary to reference the ScriptDefinition object within the HtmlTextAreaScript. If you check the documentation for HtmlTextAreaScript, you'll notice that it is a lightweight container for ScriptDefinition objects and parameter values. This is because the parameter values are saved with the HtmlTextArea itself at design time, and cannot be modified at runtime - i.e. by changing a property control or otherwise. However, using this API it is now of course possible to change the parameter values using IronPython scripting if you so desire.

You can create new HtmlTextAreaScript objects really easily, and add them to an HtmlTextArea. Extending the previous example:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

from Spotfire.Dxp.Application.Visuals import HtmlTextArea
from Spotfire.Dxp.Application.Scripting import HtmlTextAreaScript

# Cast the Visualization object as an HtmlTextArea object
textArea = textArea.As[HtmlTextArea]()

# Iterate over the scripts used in this HtmlTextArea
for script in textArea.Scripts:
	print script.ScriptDefinition.Name

sm = Document.ScriptManager
bFound, scriptDefinition = sm.TryGetScript("Accordion")

# Print the names of the parameters for reference
for parameter in scriptDefinition.Parameters:
	print parameter.Name

# There are two parameters - give them values (string only!)
parameterValues = {'parameter1': 'Hello', 'parameter2': 'World'}

newHtmlTextAreaScript = HtmlTextAreaScript(scriptDefinition, parameterValues)

# Add the script to the HtmlTextArea
textArea.Scripts.Add(newHtmlTextAreaScript)

One small caveat I found when writing this example is that if an existing script is used in the HtmlTextArea that has parameters that are not defined, iterating over the scripts will fail with an error. You can solve this by calling the Remove() method on the HtmlTextAreaScriptCollection object on the HtmlTextArea (use the Scripts property), and re-add the scripts to the HtmlTextArea, whilst defining the values of the parameters as above. This error appears to be a bug and will hopefully be rectified in future versions of Spotfire. Of course, you can also rectify this by manually assigning values to the parameters for the script in the Text Area in Spotfire Analyst itself.

There are other useful methods available on the HtmlTextAreaScriptCollection class for managing the scripts associated with the Text Area - check out the documentation!

Adding Your Own Instance of jQuery and Other JavaScript Libraries

For the background on why you would need to do this, please read How to include your own instances of jQuery and jQueryUI in Text Areas. However, you do not need to follow the steps in that article! The script management APIs will allow you to do this in a more automated fashion.

Here is an IronPython script for including jQuery and jQueryUI in all existing HtmlTextAreas:

# Copyright © 2023. TIBCO Software Inc.  Licensed under TIBCO BSD-style license.

import clr
clr.AddReference('System.Web.Extensions')
from System.Web.Script.Serialization import JavaScriptSerializer
from System.Net import WebClient
from Spotfire.Dxp.Application.Scripting import HtmlTextAreaScript, ScriptDefinition, ScriptLanguage, ScriptParameterCollection
from Spotfire.Dxp.Application.Visuals import HtmlTextArea

# Create a web client
client = WebClient()

# Get the jQuery slim, minified code
jQueryCode = client.DownloadString("https://code.jquery.com/jquery-3.6.3.slim.min.js"")

jQueryCode += "// Assign a global variable so that jQuery can be accessed by other scripts later. \n \
// Use the noConflict mechanism to restore any exising jQuery object in scope. \n \
// This avoids conflicts with any version of jQuery that might be used by Spotfire. \n \
window.CustomJQuery = window.$.noConflict(true);\n "

# Get the jQueryUI minified code
jQueryUiCode = client.DownloadString("https://code.jquery.com/ui/1.13.2/jquery-ui.min.js"")

sm = Document.ScriptManager

jQueryScriptDefinition = ScriptDefinition.Create( \
	"jQuery", \
	"jQuery Code", \
	jQueryCode, \
	ScriptLanguage.JavaScript, \
	ScriptParameterCollection([]), \
	True)

jQueryUiScriptDefinition = ScriptDefinition.Create( \
	"jQueryUI", \
	"jQueryUI Code", \
	jQueryUiCode, \
	ScriptLanguage.JavaScript, \
	ScriptParameterCollection([]), \
	True)

# Now add these scripts to the ScriptManager
sm.AddScriptDefinition(jQueryScriptDefinition)
sm.AddScriptDefinition(jQueryUiScriptDefinition)

# Add these scripts to all existing HtmlTextArea s
for page in Document.Pages:
	for visual in page.Visuals:
		htmlTextArea = visual.As[HtmlTextArea]()
		if not htmlTextArea is None:
			# Insert jQueryUI as the first (index 0) script
			htmlTextAreaScript = HtmlTextAreaScript(jQueryUiScriptDefinition, {})
			htmlTextArea.Scripts.Insert(0, htmlTextAreaScript)
			# Now insert jQuery before jQueryUI
			htmlTextAreaScript = HtmlTextAreaScript(jQueryScriptDefinition, {})
			htmlTextArea.Scripts.Insert(0, htmlTextAreaScript)

For simplicity, the above script doesn't check to see if jQuery or jQueryUI scripts already exist, but that should be a fairly simple exercise to implement, given what is discussed in the rest of this article.

Cleaning Up! Spotfire can have multiple scripts with the same name. This scenario is even more likely to occur if scripts are created using the APIs discussed in this article, particularly while experimenting! The ScriptManager.TryGetScript will take the name of a script and use it to try to find and return a script with that name. But if multiple scripts with that name exist, which one will be returned? 

A new method, GetAllScriptsWithName has been added to the ScriptManager class - the purpose of this method is to assist with the cleanup of redundant scripts. It will return an enumeration of all the scripts with the name specified, so you can iterate over them, or store them (one by one) in your own collection.

"Ah", I hear you say, but how do I check if one script is equivalent to another? The ScriptDefinition instances cannot be compared for equality as, internally to Spotfire, each script has a unique "trust" signature. So, the ScriptDefinition has the IsEquivalentTo method just for this purpose. It compares another script's script Name, Descripttion, ScriptCode, Language, Parameters and WrapInTransaction with the instance on which it is called. It will return true if the scripts are equivalent, otherwise false.

Obviously, once you have identified duplicate scripts, you can just use the Remove method on the ScriptManager to remove any unwanted duplicates. Be aware that you should probably add duplicate ScriptDefinitions to your own collection while iterating over the duplicates returned from GetAllScriptsWithName, then Remove the duplicates afterwards. It's not recommended to Remove them while iterating over them. Doing this kind of thing can lead to errors - in programming it's not generally allowed to remove items from a collection while you are iterating over that same collection. If you were allowed to do so, it could lead to all sorts of memory corruption issues.