Spotfire Copilot™

Deployment and Prerequisites

The deployment is through the installation of an SPK package on the Spotfire® Server for a seamless, custom-panel interaction. The frontend component is thus a Spotfire custom panel. The backend components for Copilot make up what we call the Orchestrator. All components are detailed in the installation guide, which will be part of the downloadable package through the Spotfire Community Exchange. The prerequisites include: access to an LLM API, a Vector Database or Azure Cognitive Search, Docker components, and Spotfire® Analyst/Server access. Below is a high level overview of the architecture of Spotfire Copilot:

 

Background on LLMs

LLMs are advanced NLP (natural language processing) models that use machine learning algorithms to analyze and understand human language. These models are trained on vast amounts of text data, such as books, articles, and websites, allowing them to generate human-like language and perform various language-related tasks. For more details, we have the following articles and whitepaper: NLP and LLMs in Spotfire and NLP glossary.

The recent advancements in artificial intelligence domains and computational power have enabled the development of larger and more powerful language models. These models have been shown to excel at a variety of language tasks, including question answering, code generation, text summarization, and multimodal advances, among others. The most common way to engage with these models is via prompting them with an engineered piece of text that results in the desired responses.

Spotfire Copilot supports most common LLMs that are cloud-based or on-premises. For simplicity, in the remainder of this article, we assume that the employed LLMs and accompanying services come from Microsoft Azure OpenAI (i.e., GPT-3.5-Turbo, GPT-4) unless otherwise mentioned explicitly. These models will be within the customer’s own Azure subscription (as opposed to Spotfire) to ensure the maximum level of data privacy and control. Note, however, that the architecture is agnostic and can run anywhere the required components can run.
 

Sample Scenarios

Through a few examples, we will demonstrate how Spotfire Copilot can help users be more efficient (for more information on the Copilot evolution, check out Spotfire Copilot: Interact with Spotfire in Human Language).
 

Watch the following for a walkthrough of an example Copilot use case:

 


Let's go through this example, step by step.

First, we open up Spotfire® Analyst and the Spotfire Copilot custom panel via the toolbar. Copilot shows an initial welcome message.

 

How to perform an operation in Spotfire

We can perform Spotfire question-answering, which provides answers with citations from additional knowledge in Spotfire documentation (think product documentation, Spotfire textbooks, etc.).

The user also has the ability to upload their own documents as additional context for the LLM. In this example, we have uploaded documentation on the topic of OEE, giving Copilot the subject matter expertise to answer domain-specific questions. Note that citations are still provided for these additional documents.

 

Explain loaded data

When wanting to learn more about the data in our dashboard, Copilot can summarize and explain its different datasets and associated columns. This functionality doesn't require the Copilot to have prior knowledge about data and their variables.

 

Create visualizations

To start analyzing the data, we ask the Copilot to show us unscheduled down over time. 

Spotfire Copilot is trained to decipher what type of visualization to create based on the user input, but we can ask Copilot to instead turn this into a bar chart.

We click the "Create Visualization" button and the chart immediately pops up on the current page. The resulting chart is a native Spotfire visualization, identical to manually-crafted ones.

We can also create more advanced charts. Here, we specify the four primary reasons for unscheduled down and ask to see these as a stacked bar chart. Note that existing markings and filters can be applied, along with the ability to filter and limit data in the visualization. Currently, Copilot is able to produce line, scatter, and bar charts.

 

Explain visualizations

Given a list a available visualizations, we one to explain and Spotfire Copilot identifies and calculates trends, outliers, clusters, anomalies, and more. This feature is particularly useful for communicating the interpretations of an analytical chart to end consumers and generating readable reports.

 

Create a data function

We can automatically create Spotfire® Data Functions with the Copilot. In this scenario, a data function might be useful for forecasting availability at this particular production plant. Not only does Copilot formulate the python script with proper comments, but it also populates the appropriate input/output parameters along with descriptions.

For further questions, inquires, or feedback you can leave comments on this post or email us at datascience@spotfire.com.

 

Extra Resources

• Download Spotfire Copilot
• Spotfire Copilot: Interact with Spotfire in Human Language (history of our Copilot work)
• NLP and LLMs in Spotfire (overview of NLP and LLM tools for Spotfire)
• NLP glossary (understanding common terminology)
• Have questions? Need support? Contact us at datascience@spotfire.com.
   

FAQ

 

The Spotfire Copilot artificial intelligence tool is a free offering licensed under the Apache License 2.0 for use with Spotfire. As the tool is not part of the commercial Spotfire product, Cloud Software Group, Inc. does not offer paid support or maintenance for Spotfire Copilot.