What's New in Spotfire® 10.0

Spotfire 10.0 represents a major mark in the evolution of Business Intelligence and Analytics tools. For the first time, a product covers all the four main paradigms of interaction with BI data, while at the same time providing the full capabilities of a modern BI platform.

Spotfire® 10.0

Spotfire® 10.0 represents a major mark in the evolution of Business Intelligence and Analytics tools. For the first time, a product covers all the four main paradigms of interaction with BI data, while at the same time providing the full capabilities of a modern BI platform. Spotfire 10 includes all consumption of BI information through dashboards, add-hoc visual analytics, canned analytic apps, predictive analytics, mobile apps, static reports and web portal mashups. With Spotfire 10 you no longer have to be restricted to one interaction paradigm, such as direct manipulation. With Spotfire 10 you can analyze your data using search, automatic insights or direct manipulation, or even modify and build your application using a workflow-based approach. The fact is that each of these user interface paradigms have their own merits and drawbacks, so you should be free to choose the one that works best for the task at hand.

But Spotfire® 10 does not stop at providing a state of the art user experience for BI and Analytics; Spotfire® 10 pushes the border of BI by letting you analyze streaming data from dozens of streaming data sources such as Kafka, MQTT, Salesforce Streaming, WITS, OsiPi and many Capital Market Exchanges, and of course other TIBCO Technologies such as FTL and EMS. With Spotfire 10, you can build real-time dashboards just as easily as you build dashboards based on static data, and Spotfire, in fact, lets you mashup streaming and historic data to give you a view both of what's happening now and how that compares with what happened historically, or it lets you bring in static data as context to the streaming data.

Read more about Spotfire® 10.0 below, and also checkout the Spotfire® X webinar series - a series of webinars covering different aspects in detail.

Search

Search for visualizations

Spotfire® 10.0 has a powerful search feature that lets you search both for columns or values in your data, to be able to show recommended visualizations, or for Spotfire® features. This makes it even easier to visualize and analyze your data the way you want to, and it makes visual analytics accessible to anyone who knows how to search.

Type, for example, "Sales per Region and Year" and Spotfire® responds with recommending the corresponding visualizations. You can choose to click or drag a visualization from the search interface to the visualization canvas.

Search for values in your data

It is also possible to search for values in your data. For example, if you are looking for the sales of a specific sales rep, the sales in a specific country, etc., you can just type the name and Spotfire® will help you mark it.

Search to drive your analysis

The search feature also lets you search for Spotfire® functionality, such as adding a calculated column or similar, or even for data stored in the library.

AI-Powered Visualization Recommendations

Spotfire® 10.0 provides visualization recommendations powered by machine learning, that help you find relationships in the data. If you select a column in the Data in analysis flyout, Spotfire® now shows recommended visualizations including also other columns that seem likely to have a relationship to the selected column. This makes it easier and faster to analyze your data.

New Spotfire® UX

The Spotfire® user interface has been redesigned, and is now visually cleaner and keeps the functionality grouped more logically.

Top Menu

The top menu has sections for File, Data, Tools, Visualizations, Help, etc., that contain access-points for many features. On the right-hand side, there are icons for filters, bookmarks, collaboration, search, etc.

Files and data

The + icon lets you browse and search for all kinds of data sources, files, data functions, data connections, custom data sources, etc., both locally and in the Spotfire® library.

Data in analysis flyout

The Data in analysis flyout lets you see details about the selected column, such as the distribution of values, max, min, unique values, etc., and access various functions for columns.

Visualization types flyout

The Visualization types flyout lets you choose a visualization type and drag it to the desired position on the visualization canvas.

Data canvas

The Data canvas is the new place to review and author the data pipeline for each data table (the Source View). It is accessed through the icon at the bottom of the Authoring bar to the left.

Visual Analytics for real-time data

By connecting to the new product Spotfire® Data Streams, or Spotfire® Streaming Live Datamart, Spotfire® Analyst can now visualize data from dozens of streaming data sources such as Kafka, MQTT, Salesforce Streaming, WITS, OsiPi and many Capital Market Exchanges, and of course other TIBCO Technologies such as FTL and EMS.

SpoSpotfire® fire 10.0 makes creating real-time applications and dashboards just as easy as with traditional data. This enables organizations to react quicker to issues and opportunities and optimize their business further. With this capability, Spotfire® 10 pushes the border of BI and Analytics into use cases that previously could only be served with expensive custom applications.

Limit streaming data to a time range

With streaming data it is possible to limit the time range shown in the visualization to the last X minute, or any other time range you want to see. To view just the last X minutes in a visualization with time on the X-axis is easily done by a click on the X-axis scale.

Note that it is also possible to limit the data from the Data section of the Visualization Properties dialog, and that there you are not limited to the last X minutes but can configure arbitrary time intervals, such as showing the data from 5 to 10 minutes ago.

Visual Analytics

Drag and drop to place new visuals on the visualization area

When creating a visualization from the visualizations flyout, Search or Recommended Visualizations, you can now drag it and place it on the canvas where you would like it. This makes it quicker to create dashboards with the layout you want.

Configurable preferred aggregation method in Spotfire® visualizations

It is now possible both to set a preferred aggregation method for individual columns, but also a globally preferred aggregation for numerical columns. This means that Spotfire® will use the preferred aggregation method instead of "Sum" at any time when it needs to default to an aggregation method in a visualization. This is useful, for example, when working with data that does not make sense to sum up, such as Temperature, Age, etc. The global setting is used unless there is a specific setting for the concerned column.

The global setting is configured from Tools > Options, in the Visualizations page, and requires Spotfire® Analyst.

For individual columns, the preferred aggregation method can be specified in the Data in analysis flyout, using Spotfire® Business Author or Spotfire® Analyst.

Add user name and/or timestamp in prepared PDF reports header or footer

The prepared PDF reports can now be configured to display the current user and/or the current time in the header or footer.

Change coordinate reference system in Spotfire® Business Author

Spotfire® Business Authors users can now configure the coordinate reference system (CRS) and projection of a map chart and its associated layers. This also benefits Analyst users with a nicer, faster UI to set CRS/Projection preferences.

Geocoding coverage updates

Over 160,000 new cities are now available worldwide.

New administrative areas are supported for US, that includes:

Area Code
Borough
CBSA
Congressional District
School District

Data Wrangling

The summary view

When adding and replacing data in Spotfire® , the summary view will not only show you how your data will be added, it will also recommend how data is added. Individual features of the summary view are described in separate sections below.

Define add columns and add rows without waiting for data to load

Since the summary view is displayed before any data is loaded, you can now define Add Columns and Add Rows operations without loading one of the data tables into memory first. Combined with previews and smart UIs, this allows you to get your operations configured right from the start, before starting to load potentially large amounts of data.

Recommended add rows, while adding data

The recommendation engine is built right into the summary view and will run across all listed tables. This allows the recommendation engine to work across tables not yet loaded into the analysis file. For example, you can let the recommendation engine union 12 Excel files with sales per month without loading them one by one. In addition to previous Business Author client support, recommendations for Add Rows is now available in the Windows client Spotfire® Analyst.

Add columns with recommended column matches, while adding data

Just like with Add rows above, you can configure Add columns in one step, before loading data. As in previous Spotfire® releases you are guided by recommended column matches and a smart web-based dialog with a preview. The Windows client, Spotfire® Analyst, now uses web UIs only, for configuration of Add Columns operations.

Recommendation engine support for links between tables

As previously mentioned, Spotfire®'s recommendation engine for visualizations is now an integrated part of the data panel.

There is also a new type of recommendation available in Spotfire® 10, recommended table links. By linking tables together, you can use shared marking between visualizations representing data from different data tables.

This recommendation provides a shortcut to create a Spotfire® relation, which is much faster than using the Data Table Properties Relations tab.

Categorization of in-db columns in the data in analysis flyout

As with in-memory data tables before, in-database data tables are now easier to navigate because their columns are automatically categorized and grouped by Numbers, Categories, Currency, Time, Location, Identifiers and Binary.

Data Access

Apache Drill support

Spotfire® now includes a native connector for Apache Drill. It's available in the Connect to list in the Windows client Spotfire® Analyst. The connector works in conjunction with the MapR Drill ODBC driver. Install the driver on your Windows client and on the Windows Server Spotfire® Node Manager to activate this connector in the Spotfire® on-premises platform.

Together with Dremio, this is the first connector that fetches the connection string details from DSNs and stores them in a Spotfire® data source object. This enables you to use all the granular settings found in the DSN configuration UI.

Dremio Support

Spotfire® now includes a native connector for Dremio. It's available in the Connect to list in the Windows client Spotfire® Analyst. The connector works in conjunction with the Dremio ODBC driver. Install the driver on your Windows client and on the Windows Server Spotfire® Node Manager to activate this connector in the Spotfire® on-premises platform.

Together with Apache Drill, this is the first connector that fetches the connection string details from DSNs and stores them in a Spotfire® data source object. This enables you to use all the granular settings found in the DSN configuration UI.

New connector list

The new way Spotfire® lists connectors makes it easier to find the data source you would like to connect to. Each connector also has an updated help text which allows you to instantly find the details you need, for example, to activate a connector by downloading and installing the corresponding driver. The new list also saves clicks. If you select the wrong connector, the list is still available so that you can switch to the connector you intended to use.

In the Apache Drill image above, we can see how two existing data connections in the library are listed together with the option to establish a new connection. This encourages reuse of data connections that are already stored in the library, instad of connecting from scratch with each analysis created ad hoc.

The new list is also searchable from Spotfire®'s updated search function (see the Search section at the beginning of this article).

Data tools consolidated into one menu

All data-related menu items are now gathered under a Data menu.

Server and Administration

Scheduling Automation Services jobs

In the new Automation Services area in the administration interface, you can schedule Automation Services jobs, and monitor the activity of all Automation Services jobs that are run in your Spotfire® environment.

The TERR service

The TERR service is now available as part of the Spotfire® environment, along with the existing Web Player service and Automation Services. This simplifies deployment, scaling out (since it uses the standard Spotfire® method for deploying nodes) and authentication (since the authentication is handled through the Spotfire® Server).

Node managers can now be installed on Linux computers to run the TERR service; see Installing a node manager (RPM Linux) or Installing a node manager (Tarball Linux).

Improved scheduled update job handling

Scheduled update jobs that cannot be immediately run are now all queued on the Spotfire® Server for distribution to Spotfire® Web Player instances as they become available. This results in a more robust routing of jobs than previously, where each service maintained its own job queue after its maximum number of concurrent updates was reached. Administrators can still set the maximum number of concurrent jobs per service; for more information, see the concurrentUpdates setting in Spotfire.Dxp.Worker.Web.config file.

You can change how often the server deletes the scheduled job history by editing the configuration.xml file; see Changing how often the scheduled job history is cleared.

Node manager performance monitoring

At the DEBUG logging level, the node manager now produces a performance.monitoring.log file that is similar to the server log file with the same name.

New commands

create-scheduled-jobs creates scheduled Automation Services jobs from a local JSON file that is created by the administrator.

remove-config-property modifies the configuration.xml file to remove the value(s) of a specific configuration property.

Developer

Preferred aggregation method

An API to get or set the preferred aggregation method to be used by plot heuristics when creating aggregated expressions from a data column.

API reference:

Default layers for map chart

An API to load the default layer, for example base map layer, TMS layer, or feature layer, when configuring a map chart.