What's New in TIBCO Spotfire® 11.0

Visualization Mods
Data Access
Google Cloud SQL support for MySQL, PostgreSQL and SQL Server
Cloudera Data Platform (CDP) Hive support
IBM Performance Server support
Data Wrangling
- Unhide hidden columns
Visual Analytics
- The f(x) flyout ? quick access to analytic tools and your favorite data functions
Server and Administration
Data Science
- Automatically distribute a custom python interpreter to Spotfire Analyst users and the Python service nodes
APIs / SDK
- Customize the welcome page

Spotfire 11.0 provides the much awaited Spotfire Mods, a new lightweight cloud-ready framework for building new interactive visualization types, called visualization mods, that can be used in Spotfire like any other visualization. By connecting mods to any Spotfire data source and integrating with data science and other built-in Spotfire capabilities, you can create highly customized analytics applications directly within the Spotfire environment.

Spotfire® 11 also introduces the f(x) flyout that gives you quick access to analytic tools and your favorite data functions.

In addition, Spotfire® 11 contains many highly requested features, including:

Amazon Redshift Ready certification and complete connector overhaul
Google Cloud MySQL, PostgreSQL and SQL Server support
Automatic distribution of a custom python interpreter to Spotfire Analyst users and the Python service nodes
Customizable welcome page

Visualization Mods

Introducing Spotfire Mods

Spotfire 11 introduces a new concept called Mods, a lightweight, cloud-ready extension framework, that gives you the ability to rapidly build shareable, reusable components that add new functionality to Spotfire and can tailor Spotfire applications to any industry and any role.

The mod concept originally comes from the gaming community, where games can be modified by any developer via a simple API, to create new levels, worlds or characters that transform the game to a new experience.

By unlocking the creativity and passion of the developer community, infinite customizations are possible, in video games as well as in Spotfire.

With this first release of the Mods framework, you will be able to build new interactive visualization types, called visualization mods, and use them in Spotfire like any other visualization.

This is useful for many different roles:

Analysts looking for insights into data, and wanting to share those insights with others, get access to a rich set of specialized visualizations, tailored to their industry and their role, from within the easy-to-use analytics environment of Spotfire.
Report developers who build interactive dashboards for executives and other business users similarly can tailor their dashboards using specialized visualizations that resonate with their audience, while simultaneously taking advantage of all data access, data wrangling and interactive capabilities of the Spotfire platform.
Web developers who build analytic applications for specific industries and business functions can use Spotfire as a platform to save time and get to market faster, instead of building applications from scratch, without sacrificing the ability to use very specific visualizations and actions that fit their end users' needs.

What is a visualization mod

A Spotfire visualization mod is a lightweight, shareable and reusable visualization component that is rapidly built with web technologies such as JavaScript and HTML. Developers have the ability to use a simple API in their code to access the functionality in Spotfire and they can easily integrate third-party JavaScript visualizations and libraries.

With visualization mods you can add new visualization types to Spotfire that look and feel like native Spotfire visualizations. Analysts, authors and consumers can interact with a visualization mod the way they are used to with Spotfire visualizations and do not need to know anything about how the mod was implemented.

A visualization mod can be reused in different analyses with different data sets, that can be in-memory data, in-database data, streaming data and data-on-demand from any Spotfire supported data source.

It is very easy to distribute a visualization mod as a simple download for the TIBCO Community or a third-party website, and it can be deployed to the Spotfire environment by an individual user for personal use or by an administrator for use by a group of users.

Visualization mods are supported in Spotfire Analyst (Windows) and Spotfire Consumer/Business Author (web) on premises, in private cloud or in public cloud (TIBCO Cloud Spotfire) environments.

Developing visualization mods

A visualization mod is built with web technologies , such as JavaScript, HTML, CSS etc., and runs in a sandboxed iframe in the Spotfire client.

The Spotfire Mods API gives you access to the Spotfire data and other functionality. It is based on TypeScript, which enables syntax highlighting and code completion in the editor.

You can easily integrate any third-party JavaScript libraries, such as D3.js, Highcharts or React, with your mod to handle the graphics or help speed up the development.

To build a Mod you need:

A running instance of Spotfire (Analyst or Business Author)
A code editor. It is highly recommended to use Microsoft Visual Studio Code when developing a mod. The mod examples on GitHub include predefined settings for VS Code that enable default build tasks and intellisense that provides inline API help.
Node.js installed to enable live preview in Spotfire as you develop.

mod-development-code.png.6d1c8d7cd751075ed7b07266f131f151.png

Learn more about how to develop visualization mods in the Spotfire Mods developer documentation on GitHub, where you will also find detailed API documentation and source code examples that can be used as templates for your own development projects.

Sharing and accessing visualization mods

A visualization mod can be reused in different analyses, with different data and configuration and by different users. You can easily share a visualization mod with others by:

saving it to the Spotfire library where others who have the right permissions can find it by browsing or searching for it from the visualization flyout
as a separate .mod file made available via email, file shares or public websites, such as the TIBCO Community
embedding it in a Spotfire analysis file.

On TIBCO Community Exchange you will find a collection of visualization mods, for different industries and use cases, that you can download for free and use in your analyses.

It is easy to deploy a visualization mod into the Spotfire environment. For example, you can just drag and drop a .mod file to your open analysis session. You can then save it to the library so it can be reused in other analyses and shared with other users. An administrator can deploy and share a mod to a group of users.

Saving visualization mods to the Spotfire library is an effective way of sharing visualization mods within a Spotfire environment, since it makes them easily accessible by all users. It also allows the users to pin mods to their visualization flyout, which means that they are added next to the list of native visualizations.

By pinning a mod it becomes instantly available to be used in other analyses and the mod will be included as an option whenever you select visualization type, for example, when you want to switch to another visualization or create a details visualization.

Pinning mods to the visualization flyout can also be centrally managed. An administrator can set up a collection of mods that the users will have access to.

Another benefit of sharing a visualization mod in the Spotfire library is that you will get automatic version handling of the mod. By default, if a visualization mod is re-saved to the Spotfire library, all instances of the visualization mod are updated in all analyses where it is used.

Using visualization mods

Visualization mods make analytics available to more business users by making it affordable for subject matter experts to build fit-for-purpose specialized interactive visualizations for different business problems.

Visualization mods also greatly expand the types of visualizations accessible to Spotfire Analysts, Authors and Consumers by enabling customers, partners and 3rd party developers to easily build and distribute new visualization types to the Spotfire community.

Because Spotfire visualization mods look, feel and behave like native Spotfire visualizations, Analysts, Authors and Consumers can easily use these new visualizations types in their analyses without any knowledge of how they were implemented.

A visualization mod is highly configurable to be reused in different analyses with different data. The configuration of a mod?s data view and appearance is done with drag-drop or direct manipulation with built-in UIs. The user has access to the standard axis selectors supporting the full Spotfire expression language. The legend and the properties panel provide configuration options for data, appearance, colors, subsets and other standard visualization properties. Mod-specific settings can be exposed through a popup panel or using a custom UI.

With visualization mods you can expect the same kind of interactive experience, with for example, marking and highlighting as you are used to with native Spotfire visualization.

A visualization mod is immediately responsive to events that occur in other parts of Spotfire. This can for example be data changes such as filtering or marking (in other visualizations). It can also be a change of the visual theme, font size or other layout changes.

A mod is also, by default, integrated with common visualization workflows, such as actions on marked data, export to PDF, maximize the visualization view, creating a details visualization and much more.

Trust model for visualization mods

Security is built into the Mods framework with a new trust model based on digital signing with certificates. By default, each user gets a personal certificate, automatically issued by the Spotfire server when creating a mod, but it is also possible to sign a mod using a certificate provided by a certificate authority, allowing the mod to be signed with, for example, a company signature.

When adding an untrusted mod, a user with proper permissions is prompted with the question of whether or not it comes from a reliable source and, if you have permission to apply trust, you will have the option to trust either that specific mod or trust all mods from that signature.

Trusting can be done by a user for personal use or it can be centrally managed, allowing administrators of a system to configure so that certain certificates are automatically trusted by all members in a specific group. For example, they can add the company certificate as automatically trusted to the group ?Everyone?, with the result that all extensions signed with that certificate will work automatically for all users. See 'Trusting custom content in the Spotfire environment' in the TIBCO Spotfire® Server and Environment - Installation and Administration manual for more information.

Governance of visualization mods

With different license configurations an organization's administrator can govern mods to fulfill scenarios like:

Only admin can save new mods to the library, but all users can use approved mods
Only a selected group of users are allowed to trust mods for themselves and save to the library, but all users can use mods signed by the company certificate
All users are allowed to trust mods for themselves

Group-level preferences for Mods allow the administrator to control what mods turn up in the visualizations flyout by default for all users.

Data Access

Amazon Redshift Ready certification and complete connector overhaul

We listened to your feedback and worked closely with the Amazon team and completely overhauled the Spotfire connector for Amazon Redshift.

Together with the Redshift Ready certification introduce support for a number of new features that enable you to do more in Spotfire with your Redshift data!

Here is the complete list:

Spotfire is recognized as Redshift Ready

As an AWS Partner Network (APN) partner, TIBCO Spotfire has been validated to follow AWS best practices and gather data insights and analytics productively, and at scale.

TIBCO Spotfire is available on AWS as software-as-a-service, as a private service and as a Turn-key AWS Instance. No matter which one you choose, or if you run Spotfire on your own servers, you can trust Spotfire to support Amazon Redshift in the best possible way.

Self-service access to Redshift Spectrum

Connect to your S3 files with a few clicks, just as if they were regular Redshift data tables. This used to require some custom SQL but now Spectrum tables are listed in the Views in Connection dialog by default, for self-service access.

Redshift spatial analytics

Bring spatial data into the in-memory data engine, and leverage the market-leading geospatial analytics features of Spotfire. The GEOMETRY data type is now supported and automatically mapped to the WKB data type, when imported into Spotfire.

Mark more items in visualizations

In visualizations where you group by multiple columns, you can now mark more items at the same time. Previously, marking queries could easily become too large with many marked items. The connector is now using tuples-in queries which means more items can be selected before reaching the maximum query size.

A comparison between Spotfire 10.10 (top) and Spotfire 11 (bottom). All records are successfully marked in Spotfire 11.

More Redshift authentication options

The previous connector had a good selection of connection attributes and settings. The overhauled connector adds new authentication methods such as AWS Profile, AWS IAM Credentials, AD FS, Azure AD, Okta and PingFederated.

Each authentication option has its own settings. You can choose to save non-secret settings in the analysis file.

Custom Redshift connection attributes

In addition to settings available via point and click, you can now enter additional connection string attributes as settings-value pairs. This ensures that Spotfire can connect to most Redshift environments in an efficient way.

Settings added to the list are verified so that they have allowed settings names.

More Redshift functions supported

All deterministic functions, including binning functions, have now been mapped, which means that you can push more calculations into Redshift using Spotfire custom expressions.

Google Cloud SQL support for MySQL, PostgreSQL and SQL Server

The native support for cloud data sources is continuously extended. The Spotfire connectors for Microsoft SQL Server, Oracle MySQL and PostgreSQL have been verified with the corresponding Google Cloud SQL databases. This includes the Google Cloud SQL proxy client which provides secure access to your instances without the need for authorized networks or for configuring SSL.

Cloudera Data Platform (CDP) Hive support

The Spotfire Connector for Cloudera Hive now supports:

Cloudera Data Platform (CDP) Data Center (CDP on premises)
Cloudera Data Platform (CDP) Private Cloud
Cloudera Data Platform (CDP) Public Cloud

IBM Performance Server support

The TIBCO Spotfire Connector for IBM Netezza has been upgraded and verified to support the next step in the evolution of Netezza, IBM Performance Server (IPS).

"IPS is a cloud-native system of insight that is 100 percent compatible with existing Netezza and PureData System for Analytics appliances. IBM Performance Server extends current Netezza speed by incorporating NVMe memory, faster processors and 64-bit technology. The result is significantly faster performance with a substantially smaller footprint." https://www.ibm.com/analytics/netezza

The data connector for IBM Netezza now also supports three new data types (INTERVAL, VARBINARY, and JSON) and a large number of new functions.

Data Wrangling

Unhide hidden columns

It is now possible to make columns that previously have been hidden (for example, when attempting to delete columns that are referenced in calculated columns) visible again by removing the Hide operations from the data canvas.

The option to hide a column instead of deleting it.

In the Data Canvas you can unhide a hidden column.

Visual Analytics

The f(x) flyout ? quick access to analytic tools and your favorite data functions

The f(x) flyout is a handy way for Spotfire Analyst or Desktop authors to easily reach their analytic tools. By default, the flyout will contain the Spotfire tools for Data relationships, K-means clustering, Line similarity, Hierarchical clustering, Regression modeling and Classification modeling, but the biggest advantage and flexibility with the flyout lies in the possibility to pin your own data functions to the flyout.

Pin a data function found in the library.

By pinning your most used data function definitions to the flyout, you will always be able to easily apply the data function to new analyses.

You always pin new data functions from the library. The search field in the f(x) flyout searches the f(x) flyout itself and not the library. However, if you click the menu to the right of the search field, there is a shortcut to the library which automatically searches for other data functions that you can pin:

Information about already used data function instances in the analysis is still available in Data Function Properties.

An administrator can push pinned data functions to user groups and their f(x) flyouts using a preference setting. This makes it easy to surface selected data functions and it saves time for end users who no longer need to search the library, with the risk of selecting the wrong data function.

Administrators pin all data functions that should be pinned for a user group and then copy the f(x) flyout information (content) to the clipboard from the top menu.

The copied information is easy to paste into the PinnedFunctionDefinitions preference for different user groups.

Server and Administration

Added support for OpenID Connect Single Logout

Spotfire support for OpenID Connect authentication was added in Spotfire version 7.8. In this release, Spotfire adds the ability to participate in Single Logout.

Spotfire can relay the information about user logout to the external application and act on such information from the external application, allowing for Single Logout (SLO).

The following three forms of SLO are supported:

RP-initiated Logout. See the OIDC draft for more details.
Front-Channel Logout. See the OIDC draft for more details.
Back-Channel Logout. See the OIDC draft for more details.

Note: These specifications can be used separately or combined.

Note: SLO is supported both in a single sign-on scenario (where the user authentication is initiated by an external application) and when the authentication is initiated from Spotfire.

Added commands for easier configuration and automation of OpenID Connect settings

Added the command update-oidc-provider to update an existing OpenID Connect provider configuration. This command complements the previously existing command config-oidc, for configuring authentication against one or more external providers using OpenID Connect.

Added the command register-oidc-client to register the Spotfire Server as an OpenID Connect client with an OpenID Provider that supports Dynamic Client Registration in an automated fashion.

Updates to Linux RPM packages and services

The Spotfire Server and node manager RPM packages have been reviewed to align with RPM packaging standards:

services use Systemd instead of SystemV when supported.
the installation packages automatically create a dedicated spotfire user and spotfire group.
the ownership of installed files have been reviewed, assigned to spotfire user and spotfire group.
the permissions of installed files been reviewed and restricted.
Spotfire server service (tss) runs now as spotfire user instead of root.

Note: Node manager rpm, tar.gz files and service name have been renamed from "tss-nm-<version>" to "tsnm-<version>".

Improved scaling for streaming visual analytics in Spotfire Web Player

The Spotfire Web Player can now be configured to gracefully throttle the rendering of visualizations with streaming data, based on the Web Player CPU load. This means that the utilization of the Web Player's CPU can be managed more efficiently and more users can be served real-time dashboards from the same Web Player instance.

Practically it works by monitoring the CPU load of the web player server and adjusting the interval at which streaming data is refreshed, and therefore how frequently visualizations based on streaming data update.

If the CPU load is higher than a configurable percentage value, the streaming data refresh interval will be set to a user configurable value in milliseconds.

This is configured in the Spotfire.Dxp.Worker.Host.exe.config file, and the configuration setting is called StreamingDataThrottlingRefreshIntervals.

Example:

<StreamingDataThrottlingRefreshIntervals>
    <string>20, 3000</string> 
    <string>40, 10000</string> 
    <string>60, 20000</string> 
</StreamingDataThrottlingRefreshIntervals>

With the above configuration, when the CPU load is below 20, visualizations will be rendered as often as possible. If CPU load it increases over 20%, the visualizations will be rendered at most every 2 seconds, at 30% every 3 seconds, at 40% every 5 seconds and above 60% every 10 seconds.

In the chart above there are two runs with streaming data where the blue one is without using the Throttling, and the red one using Throttling with the example settings. One can see that the CPU load without throttling increases to 80 percent and stabilizes there, and in the case with Throttling it stabilizes around 40% CPU load.

Web Admin UX improvements

We are rethinking data-flows and user-flows. We are updating some UI components.

The Web Admin UI is now, in general, a more responsive, performant, scalable and resilient web application.

Some improvements are for example:

Scheduling & Routing
- easier flow for creating an analysis routing rule with a custom schedule (that can be created & saved directly)
- easier filtering of Scheduled Updates activities with improved start & end time and time zone selection
Nodes & Services
- improved usability and stability of auto refresh
- easier working flow for managing resource pools and their resources
- empty resource pools are now allowed
- simpler navigation: resource item attributes name, service & host are linked and redirect to their details view

See the Release Notes for more details.

Data Science

Automatically distribute a custom python interpreter to Spotfire Analyst users and the Python service nodes

It is now possible to build a Spotfire .spk file that contains a python interpreter and have that automatically distributed to all Spotfire Analyst users and to the server machines running the Spotfire service for Python. This makes it easy to ensure all users have a consistent environment for executing Python data functions.

APIs / SDK

Customize the welcome page

The white-labeling mechanism of Spotfire (also known as co-branding) has been extended so that the Spotfire welcome page content can be replaced by custom content such as graphics, text and links. This extends the possibility to brand Spotfire as a domain or organization-specific analytics tool.

The arrow in the image above illustrates the area that can be customized.

The content of the custom welcome page is provided through a custom HTML file, and it is possible to insert images, text and links.

In addition, it is now possible to change the side image in the "splash screen" shown when starting the installed Spotfire client.

The image above indicates the images in the login screen that can be customized.

The Spotfire Package Builder is used to wrap the custom HTML in a Spotfire .spk, which can then be distributed through the Spotfire server's package deployment mechanism. The custom HTML is shown instead of the Spotfire welcome page.

Sign In