Process Control Monitoring and Alerting Solution

Introduction

We'd like to introduce a new solution - one that easily scales to monitor large numbers of processes and parameters and produces automated alerts when anomalies are detected.  It employs univariate Statistical Process Control methods and is an important Quality Control tool.  This method is used to monitor, control and improve any measured process.  The focus is on reducing variability to improve quality.

View a demo of the solution:

A number of versions of this solution are available and can be downloaded from the Spotfire Exchange here. If you are interested in implementing this solution, including capabilities shown here but not yet available on the Exchange, please contact us at datascience@spotfire.com.

Control Charts:  Brief History and Importance in Industry

Control Charts have played a vital role in the evolution and success of the global manufacturing industry. The control chart was first developed in 1924 by Walter Shewhart at Western Electric.  It was initially used to improve the reliability and audio quality of the candlestick telephone. It was then used during WWII by the US Army, but was not adopted by US industry after the war. However, it was introduced to Japanese industry and became an important quality tool that helped Toyota, Sony and others capture the global auto and electronics industries. This contributed to the 1980s US Quality crisis, resulting in the re-introduction of statistical process control methods in the US. It was first championed by Ford, with their Quality is Job 1 initiative, and Motorola, with 6 Sigma. 

Today, it is widely used across the globe and considered an essential tool in the mass production of manufactured goods. It's usage is growing in many other sectors, as well, such as Energy, Healthcare, and Telco.             
 

Solution Overview

This solution is quite comprehensive - many types of standard process control charts, alerting rules and charting options are supported. While rich in features, it is designed to be easy to configure for large numbers of parameters. 

The solution consists of two components. The end-user visualizes and interacts with all results in Spotfire dashboards. Spotfire® Data Science is the underlying backend computation engine, where all the QC charts are computed on a scheduled basis. Together with computation, in the backend, there is also automated batch monitoring and alerting. Hundreds to thousands of parameters can be monitored. Once an alert is triggered an e-mail can be sent. Alerts can also be investigated interactively in the frontend Spotfire dashboard application.

Download the Spotfire part of the solution from the Spotfire Exchange here.  
 

Spotfire dashboard summary page example

Here we are monitoring multiple parameters separately for product A and product B

In the upper left corner, we see the current unacknowledged alarms. This tells us where there are problems, that triggered alarm notifications, but have not yet been addressed. When we have many parameters, we need some indication about which charts currently show out-of-control or rules violations. On the bottom left, the heatmaps show all recent alarms including those that have already been acknowledged. In the application we are looking at two types of violations:

• out of control (OOC) = sample point is outside the control limits on the QC chart
• runs rules violations indicate systemic problems even if samples are not yet out of control.  For example several samples in a row have increasing values or several samples are on the same side of the center line

On the right, there is a summary of the monitored parameters. Spark lines show when the alarms took place. The Application is interactive, so if we want to look at a current alarm, we can mark a parameter and see it marked in all other visualizations on the page.

The purpose of this page is to see the overall situation and to help pick the parameter we want to investigate further. When we mark it here, the backend engine makes all calculations required to display that chart.

Control Charts

On the Chart pages, QC charts for the selected parameter can be seen. This is an X-bar chart and S (variance) chart. Points that contributed to runs rules violations are highlighted in yellow. On other pages we can see more detailed info for the X-chart, variance charts or runs violations. 

On the Runs Test page, the tests and violations are shown.  

Click on the image to view it fullscreen. Use browser back button to return to this page.

The configuration file

In this solution every parameter can have its own chart configuration.  The configuration includes:

• Chart type: X-bar R/S, MA (moving average), Exponentially Weighted Moving Average (EWMA) or Custom
• Control or warning limits display can be based on calculations from the data or predefined values you provide
• Spec limits display
• Runs rules to be used
• other configurations are defined in the solution documentation  

A configuration table defines how the charts and alarms are configured. A few rows and columns of the configuration table are shown at the top of the screenshot below.  Under the hood there is a parameterized data function calling Spotfire® Statistica (the backend computation engine) calculating charts and out of control points based on values in the highlighted row in the configuration file.

With this approach, you only need to maintain the configuration file and everything else is handled automatically in the dashboards and calculation engine.

The Computation Engine

Computations are done using visual workflows, leveraging the same configuration file for computing limits, out of control and rules violations that is used by the Spotfire data function. There is a loop implemented, shown below, that will run QC charts for all the parameters in the configuration file. This simplifies the task of implementing hundreds of QC charts. Inside the loop the results are saved into a database, so they can be picked up by the Dashboard application at any time.

In addition, the monitoring and alerting server is leveraged. It runs at scheduled intervals to determine if there are any alerts generated. It will also send email notifications in case of alerts.

This solution enables the user to review the state of alerts, investigate any parameter interactively and acknowledge the alarms. The essential component for simple implementation is one configuration file for all QC charts. Spotfire® Statistica has a data entry functionality that can be used for governed insertion and modification of current entries in the data configuration file, if needed.

Summary

This process control monitoring and alerting solution features hierarchical monitoring and alerting based on quality control charts. The Configuration Table makes it easy to set up for large numbers of parameters.  Many types of charts and rules are supported. Be aware that the example from the screenshots is only one possible implementation. Thanks to flexibility of Spotfire, simple configuration table and implementation of underlying computation engine based on visual workflows, the solution can be easily modified, customized and extended. 

• A number of versions of this solution are available and can be downloaded from the Spotfire Exchange here.  
• Try out an interactive demo yourself in the Spotfire Demo Gallery.
• If you are interested in implementing this solution, including capabilities shown here but not yet available on the Exchange, please contact us at datascience@spotfire.com.

Advanced Solutions

This solution involves batch processing of univariate statistical process control charts. While this is an essential first step in any process control program, it is really only a starting point. It is the most fundamental form of anomaly detection. 

For many applications, more advanced anomaly detection methods are required. 

• For complex processes and products, multivariate methods such as multivariate statistical process control or AI-based anomaly detection may be required to detect more subtle anomalies that degrade product quality and reliability. 
• If you need to detect problems as soon as they occur, in order to take immediate corrective action, you may need 'real-time' processing on 
•  or at the 
  . 
• Advanced Process Control solutions involve predicting the output of a process, while it is still 'in-flight', and adjusting the process recipe to hit the desired target. This may be necessary to reduce the variability of processes that are not capable (have low CpKs).
• Detecting anomalous time-series traces coming from connected sensors requires specialized data connection and pattern detection algorithms. 

Spotfire works with customers to develop and deploy advanced solutions in all of these areas.  For more information about implementing one of these solutions, contact us at datascience@spotfire.com.