Spotfire Copilot for Statistical Process Control Monitoring

Overview of Spotfire Copilot

Spotfire Copilot™ is a free, natural language extension to the Spotfire® platform. It leverages large language models (LLMs) to augment business intelligence and artificial intelligence, all in Spotfire. 

This release is custom-built to perform the following tasks in Spotfire: 

• Question-answering for Spotfire questions
• Question-answering for user-loaded documents
• Auto visualization generation and modification
• Data function generation
• Explanation of visualizations
• Data interrogation

For more information on Spotfire Copilot, check out this community article.

Application Overview

In the Spotfire Statistical Process Control Monitoring application, one can easily monitor large numbers of processes and parameters and produce automated alerts when anomalies are detected.  It employs univariate Statistical Process Control (SPC) methods and is an important Quality Control tool.  This method is used to monitor, control and improve any measured process, with a focus on reducing variability to improve quality. To learn more about this SPC solution, check out this article.

Demo Walkthrough

In this demo, I’d like to expand this application to also look at Cpk values for each parameter. We don’t have the Cpk values yet, but with Copilot, we can quickly calculate them, create visualizations to display them, and efficiently report back to our team. Without Copilot, this could be a whole day’s worth of work.

To start, I want to learn more about this metric. We have uploaded a handbook on pharmaceutical manufacturing, and by using retrieval augmented generation (RAG), Copilot can quickly give us more context in a written form, directly within the application.

I also want some additional context on what is considered a good and bad Cpk value.

Now, let’s actually calculate our Cpk values. I want to create a data function to do this, but creating one from scratch could take me 30-60 minutes, maybe even more, depending on how much domain knowledge I have, how familiar I am with developing data functions, and more. With Copilot, we speed up and simplify this process greatly.

Great! Now we have a new data table called Cpk Results containing our calculated Cpk values. Let’s move on with analyzing this new table. Perhaps I’m interested in what parameters have the highest Cpk values and what parameters have the lowest Cpk values. 

Now that we know the high and low ends of the Cpk distribution, it would also be beneficial to visualize all of the Cpk values and have a visualization that’s easy to understand. Copilot can quickly create this for us. By default, bar charts are created with vertical bars in Spotfire. Because Copilot retains conversational history, I can also modify the visualization by speaking to Copilot, rather than figuring out where in the settings I can find what I’d like to change.

I’m going to add some additional elements to this, such as color coding the bars based on where their Cpk values fall. From what we learned before, anything above 1.33 is considered good; those will be green. Anything below 1 is considered bad; those will be red. And anything in between is acceptable; those will be yellow. With my new visualization, I’m going to have Copilot help me write up a validation report. Instead of starting from scratch, I can ask Copilot to explain a visualization to me, even the one that we just created, and use that response as a starting point for my report.

To wrap this up, let’s say I want to export this page of my analysis to a PDF for my team. Instead of digging through the Spotfire documentation to see how to do this, I can ask Copilot how to do this instead.

In conclusion, we expanded this application to include Cpk values, and with Copilot, we were quickly able to calculate our Cpk values, ask questions about the new data, create visualizations and draft a validation report, and export the results for our team.

Demo Recording

Check out the video below to watch the full demo!

Resources

• Spotfire Copilot Landing Page + FAQ
• Download Spotfire Copilot
• Spotfire Copilot: Interact with Spotfire in Human Language (history of our Copilot work)
• NLP and LLMs in Spotfire (overview of NLP and LLM tools for Spotfire)
• NLP glossary (understanding common terminology)
• Have questions? Need support? Contact us at datascience@spotfire.com.