What's New in TIBCO Spotfire® 11.4 LTS

Table of Contents

• Data Science
• Amazon SageMaker Autopilot template
• Python Tools and Packages proxy support
• Configuration of alternate Python package repository via Preferences
• The Spotfire package available for download on PYPI
• Updated Python interpreter
• Data Wrangling
• Parallel data loading and compute
• Up-front prompting
• Performance gains and FAQ
• New data loading progress dialog and copy-to-clipboard
• Instant overview and easy access to settings when adding data
• Data Access
• SharePoint Online Lists single-tenant support
• Databricks SQL Analytics support
• Load up to 20 million rows in TIBCO Cloud Spotfire
• Amazon RDS Service Ready for Business Applications
• Certified Cloudera Impala & Cloudera Hive integrations
• MySQL, PostgreSQL and Amazon Redshift performance improvements
• Visual Analytics
• Multi-layers Trellis on Map Chart
• Enable/Disable Geometry Simplification
• New Save to Library Experience
• Copy links, rename and delete assets from the library browser
• Administration
• Python service is available in TIBCO Cloud Spotfire Enterprise
• New commands for migration of Spotfire database between supported database servers
• Configure the maximum number of concurrent sessions per user
• Mods / APIs / SDK
• Improved performance for Visualization Mods
• APIs for Viewing/Editing mode of an analysis
• Removed dependency on 3rd party cookies and HTTPS in Visualization Mods

TIBCO Spotfire 11.4 LTS is a long term supported release giving enterprise customers access to all the latest Spotfire capabilities. Mods - the secure, lightweight, pliable components that make Spotfire infinitely customizable,  about a dozen new visualizations in our ever expanding catalog of community Mods, multi-layer trellising in map charts and significantly faster data loading.

Mods was introduced in TIBCO Spotfire 11.0 and has been updated regurlarly with new features and improved documentation. Learn about how to access, use and develop mods on the Spotfire Mods Overview page.

Data Science

Amazon SageMaker Autopilot template

Amazon and TIBCO has teamed up on the joint mission of democratizing ML. Amazon SageMaker Autopilot automatically trains and tunes the best machine learning models for classification or regression, based on your Spotfire data, while allowing you to maintain full control and visibility.

The new Spotfire SageMaker Autopilot template data function (available for download here) makes it easy to integrate your analysis files with SageMaker Autopilot.

When you have applied the SageMaker Autopilot template to your use case, you point to a Spotfire data table and select the target column to predict. This target can be a number (such as a house price, called regression), or a category (such as spam/not spam, called classification).

SageMaker Autopilot will automatically explore different solutions to find the best model.

The result is returned to your Spotfire analysis as calculated columns, which means you can visualize the result by simply creating a chart on top of the updated data table.

If the visualized result needs improvements, Amazon SageMaker Studio allows you to further improve the model quality.

You can rerun the model at any time directly from Spotfire.

The template supports both model discovery using Autopilot, which often takes some time to complete (batch user experience), and already deployed models which return scores much faster (interactive user experience).

Python Tools and Packages proxy support

Many large Enterprise customers of Spotfire do not allow direct internet access from their corporate computers, but are instead configured to route all HTTP traffic through a central proxy server.

These proxy settings are generally pre-configured on users? computers, and the Spotfire HTTP client automatically detects and uses those, if configured. However, Python Tools calls out to the Python-native ?pip? module for installing and managing packages, which does not have any auto-detection for proxy servers.

Starting with Spotfire 11.4, you should notice that in most cases can list, search and add Python packages even though your organization is using an Internet proxy.

You should be able to search and add packages from pypi.org even if you are using Spotfire from behind an Internet proxy.

There may be cases where the Spotfire administrator wants to override the auto-detected proxy settings, or disable the automatic proxy detection. Administrator overrides are available in a new preference in Administration Manager.

Configuration of alternate Python package repository via Preferences

You may maintain an internal Python package repository (for example, in Artifactory) rather than using the public pypi.org package repo. The underlying pip Spotfire is using for package management has support for alternate repositories, and now an administrator can specify this in Preferences for all Analyst users, rather than having to set it manually on each computer.

The Spotfire package available for download on PYPI

The Spotfire package is now a package available on PYPI and can be downloaded by using the standard pip command. This change makes creating packages for distribution simpler. If you are using your own Python interpreter, you can simply install the 'spotfire' package using pip instead of extracting the wheel from the distribution and installing it.

Updated Python interpreter

The Python interpreter included with Spotfire has been updated to version 3.8.10.

Data Wrangling

Parallel data loading and compute

To take advantage of the TIBCO Spotfire columnar in-memory data engine, data has to be ingested into Spotfire. This is referred to as in-memory analytics. Prior to Spotfire 11.4 LTS, data was ingested sequentially, one data source after the other like in the illustration below.

Once all data tables were ingested, joins, unions and data transformations were executed.

Starting with Spotfire 11.4 LTS, data is loaded and computed in parallel when opening an analysis file according to the illustration below.

If you already had a simple fast file - just enjoy the pretty and more understandable progress dialog, see next 11.4 LTS feature below. The main win with this feature is for analyses with data from multiple data sources.

You can turn off parallel data loading entirely under Tools > Options > Document >Data Loading or, just for a single document, through File > Document properties > Compatibility Settings. An administrator can also control the preferences for a group in Administration Manager > Preferences >

Application > DataImportPreferences.

Up-front prompting

If you have an analysis file using prompts, you now get all (non-error) prompts up front. This means that after the prompts have been filled out, you can take a coffee break while the data loads (a bit faster), without being on stand-by for prompts along the way.

Performance gains and FAQ

This article answers the most frequently asked questions when it comes to parallel data loading.

https://community.spotfire.com/s/article/tibco-spotfirer-parallel-data-loa...

New data loading progress dialog and copy-to-clipboard

The new progress dialog shows more details than before. This makes it easy to know what you, as an end user, is currently waiting for.

The new progress dialog.

The new copy-to-clipboard feature captures all the new details, and more, to the clipboard and by simply pasting the clipboard data table into Spotfire you can analyze the data loading and computation operations. This allows you to find time-thieves and optimize performance even further.

Progress data loaded via the clipboard and visualized.

Instant overview and easy access to settings when adding data

The Summary View now expands the first data table by default. This makes it easy to check the default settings, for example, a data function's output. It also makes it easy for new users to know that, for example, data can be transformed as part of the data source. If anything needs to be adjusted the card is already expanded so it's quick and easy to do the change.

Data Access

SharePoint Online Lists single-tenant support

The TIBCO Spotfire Connector for SharePoint Online Lists now supports single-tenant SharePoint Online setups and authentication endpoints. Organizations who use their own authentication endpoints can now configure those in a new preference which is distributed to all clients.

Your tenant ID is set together with the client ID and client secret in the Azure AD Authentication preference in Spotfire Administration Manager.

With the preferences you can configure, per Spotfire user group, which Microsoft Azure registered app Spotfire should use in connections to SharePoint Online.

The preferences also take effect when you use the Spotfire installed client.

Note: You can also add your tenant ID to TenantId in the service configuration Spotfire.Dxp.Data.Adapters.MicrosoftCommon.Properties.Settings.

Important: If you have configured both the service configuration and the user group preferences for the logged in user, the preferences take precedence.

Note: If you haven't set the tenant ID in either the service configurations, or the preferences in the Administration Manager, you will automatically use the multi-tenant approach.

Databricks SQL Analytics support

The TIBCO Spotfire connector for Apache Spark SQL now supports Databricks SQL Analytics. You can now perform self-service visual data discovery on data lake data using live pushdown querying in Databricks clusters which have SQL Analytics enabled.

Load up to 20 million rows in TIBCO Cloud Spotfire

Data volumes are continuously increasing and as a user of TIBCO Cloud Spotfire you can now analyze up to 20 million rows in-memory. This unlocks new use cases and insights as it enables you to work with more of your data without the need to decrease the data volumes before loading data into TIBCO Cloud Spotfire.

As a Spotfire user on premises, you don't have this limitation and, as always, you control your deployment and any optional limitations you set.

Amazon RDS Service Ready for Business Applications

TIBCO is confirmed as an Amazon RDS Service Ready for Business Applications.

This means that when using TIBCO Spotfire in combination with Amazon RDS, as a data source or as the Spotfire Server database, you are using a certified integration.

Certified Cloudera Impala & Cloudera Hive integrations

The TIBCO Spotfire integrations for Impala and Hive are certified towards the Cloudera Certified Software for BI/Analytics Program. This means that Cloudera recognizes TIBCO Spotfire to provide a robust integration with Cloudera Impala and Hive.

https://www.cloudera.com/partners/partners-listing.html?q=tibco

MySQL, PostgreSQL and Amazon Redshift performance improvements

In the TIBCO Spotfire MySQL connector, the NO_CACHE, FORWARD_CURSOR and PREFETCH options are now used to improve import performance.

The UseDeclareFetch option is now enabled in the TIBCO Spotfire PostgreSQL connector to improve import performance.

To improve performance when you import data from Amazon Redshift, the option SingleRowMode is now enabled by default. You can override this by adding theSingleRowMode option in the Additional settings table and setting the value to 0.

Visual Analytics

Multi-layers Trellis on Map Chart

Trellised maps are used commonly to recognize similarities or differences between categories in the data in a spatial context and we just made this capability even more powerful! The trellising of a map chart can now comprise more than one layer of data. This makes it possible to compare multiple measures or categories of data within a single geographical or diagrammatic map.

Enable/Disable Geometry Simplification

Spotfire uses geometry simplification techniques to facilitate the rendering of detailed features on spatial maps, while preserving essential shapes, but for rectangle features that are typically represented by 4 or 5 points, the simplification might cause rendering inconsistencies.

Spotfire strives for the highest accuracy and precision possible when rendering features and that?s why you can now specify the number of points below which geometry simplification will not be used, when rendering features with less than 6 points.

New Save to Library Experience

A new unified experience when saving analyses to the Spotfire library has been developed. You will now see the same UI and style when saving as when you are browsing the library in the Spotfire Files and data flyout, both for Spotfire Analyst and web clients.

Copy links, rename and delete assets from the library browser

It is now possible to rename and delete items in the Spotfire library directly from the Files and data flyout. You can also copy links to library items to share them with others. To access the new options, right-click a library item in the Spotfire library section of the Files and data flyout.

Administration

Python service is available in TIBCO Cloud Spotfire Enterprise

The TIBCO Spotfire Service for Python is now available in TIBCO Cloud Spotfire Enterprise.

With TIBCO Cloud Spotfire Enterprise you can take full control over every aspect of your environment, but running it in TIBCO's private cloud environment.

New commands for migration of Spotfire database between supported database servers

The two new commands migrate-db and migrate-action-db facilitate the migration of the Spotfire configuration database and action database from a supported database server to another supported database server (e.g., from Oracle database to PostgreSQL).

Note that the commands support migration within same Spotfire version (e.g., from Spotfire Server 11.4 using Oracle to Spotfire Server 11.4 using PostgreSQL).

Configure the maximum number of concurrent sessions per user

A user can establish sessions up to the configured maximum number of concurrent sessions per user.

Further login attempts by the same user will fail (or force to logout the oldest session, as configured) when the user has reached the maximum number of simultaneous sessions allowed.

This feature covers mainly 2 different use cases:

• For security management, some customers need to restrict access to Spotfire to a configured maximum number of simultaneous session (e.g., due to US Government Security requirements). This is aligned with the NIST Risk Management Framework - Access Controls (also part of FedRAMP Security Controls).

• For license management, some customers may want to stop their end-users to evade license purchasing by sharing user accounts from different simultaneous locations.

Mods / APIs / SDK

Improved performance for Visualization Mods

The performance of visualization mods has been significantly improved. The transfer of data (rows and hierarchies) is now at least 2x faster for large data views and on par with the performance of native visualizations. Also, interactions are much more responsive with faster cancellation of obsolete API calls, for example, when undoing an operation or performing a new marking or filtering operation before the previous one has been completely processed.

APIs for Viewing/Editing mode of an analysis

It is now possible to call the API to find out whether an analysis is currently in viewing or editing mode. This makes it possible for the developer to adapt custom UIs to show or hide authoring capabilities depending on the mode, and achieve the same behavior as native UIs, such as axis selectors, icons in visualization title bars, and panels.

This API addition applies to the Mods API v1.2, the C# Custom Visualization API, and the JavaScript API for mashups, and works for installed Spotfire clients as well as web clients.

In Editing mode, the user can interact with axis selectors and property panels to configure a visualization mod:

In Viewing mode, the standard authoring capabilities are disabled, and this can now be applied to custom authoring capabilities as well:

Removed dependency on 3rd party cookies and HTTPS in Visualization Mods

The major browsers are moving in the direction of blocking third-party cookies. Safari already blocks all such cookies by default and both Chrome and Firefox have announced that they will be phasing them out. To make sure that mods will work in all browsers, we have removed all dependencies of third-party cookies. 

Static resources in visualization mods are now treated as publicly available resources, without the use of cookies. This means that there is no longer a requirement for the Spotfire Server to use HTTPS and SameSite=None, to be able to run visualization mods in web clients.