Introduction

The purpose of this community article is to help you troubleshoot issues with the verification of certificates in Spotfire. Certificate verification is used in the trust mechanism for visualization mods and external actions.

The trust mechanism in Spotfire is based on digital signatures so that Spotfire can keep track of who has created a visualization mod or an external action. This is important since mods and external actions access the data of the users and execute code created by the developer of the mod or external action. When Spotfire verifies a digital signature it needs to verify the certificate of the author who has signed the mod.

The verification of a certificate may fail due to various reasons, and this article aims to make it easier to find the cause.

The rest of this article is organized as follows:

In Section Common symptoms we list a common set of issues based on their symptoms and give some guidance on how to troubleshoot. This may be a good starting point  if you are not too familiar with certificate issues and have a concrete problem.

In Section Root causes and recommended solutions we discuss the root causes of common problems and what you can do about them.

In Section How-to: Troubleshoot using Windows certificate viewer we describe some basic troubleshooting.

In Section How-to: Collect information when reporting a support case we list troubleshooting information that is helpful if you open a support case.

We also have a set of How-to sections with hands-on instructions which we refer to from the other sections.

Common symptoms

In this section, we will list some common issues based on their symptoms and give some guidance on how to troubleshoot, depending on the symptoms of the issue. The section is organized based on where the symptom occurs.

Common symptoms in the Clients

In this section, we discuss symptoms that show up in the clients.

Invalid Signature

If a yellow warning triangle is shown with the message "Invalid signature" then it is most likely an issue with certificate verification.

The most common symptom is that a visualization mod is not displayed.

Note that the message "The visualization mod is not trusted" does not mean that it is an issue with the verification of the certificate unless you see the message "Invalid signature".

If Spotfire shows the message "Invalid signature"  together with a red error icon then the certificate is not valid or has been revoked. In that case, you should not use the mod. The yellow warning triangle, on the other hand, means that Spotfire has not been able to verify the certificate. If so the certificate may still be valid.

The message "Invalid signature" may show up in several contexts. For example, if you add a new visualization mod to the analysis or if you attempt to execute an external action. It is also shown in the 'Manage trust' dialog.

To get more information about the issue, click on the yellow triangle.

Note that you can download the certificate that is used to sign the mod in this UI. The certificate is often needed when troubleshooting.

There are many possible error messages that may be shown. Below we will cover some common ones.

Error message: The root certificate is not trusted. The entity might be signed by a user in another system.

A common problem is that the mod is signed by a user in another Spotfire system. You will then see the following error message.

We discuss how to address that case in the Section Signed by a Spotfire user.

Error message: The root certificate is not trusted.

There are multiple reasons why Spotfire displays this error message and the message may sometimes be misleading. One reason is that the root certificate is not trusted. However, it may also be the case that Spotfire was not able to build the entire certificate chain from the certificate (used to sign the mod) to the root certificate. The latter is, by far, the most common reason so we will discuss that first.

Intermediate certificates

In some cases, Spotfire needs to have access to the internet in order to fetch intermediate certificates that are needed to build the certificate chain. If you are using the web client then the problem is most likely that the Node Manager machine has no direct internet connection. If you have configured a proxy server then it may be an issue with the configuration of the proxy server or the configuration of Spotfire. We discuss this in the Section Lack of internet connection.

It may also be an issue with the caching of the intermediate certificate. We discuss that in Section Section Issue with caching of intermediate certificates when using Kerberos and Issue with caching of intermediate certificates.

Root certificate is not trusted

In the unlikely event that it is not an issue with internet connectivity then the reason for the error may be that the root certificate is not a trusted root certificate in the Windows certificate store. 

Warning: You should not address that issue by importing the root certificate into the Windows certificate store. If you import a root certificate to the Windows certificate store as a trusted root certificate you may compromise the security of your computer.

It may be the case that the certificate has been forged and that the root certificate appears to have been issued by a public certificate authority even though that is not the case.

If you are using a private certificate authority then see Section Signed with a certificate issued by a private certificate authority.

If the root certificate is issued by a public certificate authority then it will normally be in the Windows certificate store. If the root certificate has been issued recently then it may be the case that it is not in the Windows certificate store. However, if Spotfire has internet access it should be able to build an alternate certificate chain to an older root certificate.

Error message: It is not possible to determine whether the certificate has been revoked.

Another common problem is that it was not possible to check whether the certificate was revoked or not. You will then see the following error message.

To check whether a certificate has been revoked, Spotfire needs to connect to the certificate authority that has issued the certificate.

If the mod has been signed by a user in another Spotfire system, then that Spotfire system is the certificate authority. In that case, the server for the other Spotfire system needs to be accessible. To see if the mod has been signed by a user in another Spotfire system, you should look at Certificate issuer: If the certificate issuer is 'TIBCO Spotfire Code Trust Root CA'  then the mod has been signed by a user in another Spotfire system. See Section Revocation checks.

In the example above, the certificate has been issued by the public certificate authority DigiCert. To reach a public certificate authority Spotfire needs to have access to the internet. If you are using the web client then the problem is most likely that the Node Manager machine has no direct internet connection. If you have configured a proxy server then it may be an issue with the configuration of the proxy server or the configuration of Spotfire. We discuss this in the Section Lack of internet connection.

Error when trusting

Another common issue is that you get an error when you trust a mod even though the signature is valid.

The reason for that error may be that the Spotfire Server is not able to verify the certificate. If so, the Spotfire Server logs will contain the message "Error verifying the certificate path". Note that this is logged on the DEBUG level on the Spotfire Server.

The verification of certificates on the Spotfire Server can be disabled without any security implications. See Section How-to: Disable certificate validation on the Spotfire server

This issue is most likely caused by having no internet connection on the Spotfire Server. See Section Lack of internet connection if you do not want to disable the verification of certificates on the Spotfire Server.

Slow to save a mod to the library

Another problem may be that it is slow to save a mod to the Spotfire library. The most likely reason is that the Spotfire server is unable to connect to the certificate authority when it attempts to verify the certificate. It may take a long time before the attempt to connect times-out. If you see the log message "Error verifying mod signature" in the server logs, it means that the server has failed to verify the signature.

The verification of signatures on the Spotfire Server can be disabled without any security implications. See Section How-to: Disable certificate validation on the Spotfire server.

This issue is most likely caused by having no internet connection on the Spotfire Server. See Section Lack of internet connection if you do not want to disable the verification of certificates on the Spotfire server.

Common symptom in the Admin UI

When you import a certificate through the group page in the Admin UI you may get the error "The certificate path could not be verified".

The reason for this error is that the Spotfire Server is unable to verify the certificate.

The verification of certificates on the Spotfire server can be disabled without any security implications. See Section How-to: Disable certificate validation on the Spotfire server.

This issue is most likely caused by having no internet connection on the Spotfire Server. See Section Lack of internet connection if you do not want to disable the verification of certificates on the Spotfire server.

It may also be the case that the certificate belongs to a user in another Spotfire system. If so you need to configure Spotfire to accept signatures from the other Spotfire system. See Signed by a Spotfire user.

Finally, the certificate may be issued by a private certificate authority. See Section Signed with a certificate issued by a private certificate authority.

Common symptom in the CLI

You may get the following error when you import a certificate through the CLI command import-code-signing-certificate to the Spotfire Server database.

The reason for the error "Error uploading certificate: Error verifying the certificate path" is that the Spotfire Server is unable to verify the certificate.

The verification of certificates on the Spotfire Server can be disabled without any security implications. See Section How-to: Disable certificate validation on the Spotfire server.

This issue is most likely caused by having no internet connection on the Spotfire server. See Section Lack of internet connection if you do not want to disable the verification of certificates on the Spotfire server.

It may also be the case that the certificate belongs to a user in another Spotfire system. If so you need to configure Spotfire to accept signatures from the other Spotfire system. See Signed by a Spotfire user.

Finally, the certificate may be issued by a private certificate authority. See Section Signed with a certificate issued by a private certificate authority.

Root causes and recommended solutions

In this section, we will describe some common root causes for issues with certificate verification issues in Spotfire. We will also provide some recommendations on how to troubleshoot and resolve the issues.

This section is somewhat technical and if you have a concrete symptom then you may want to first browse through Section Common symptoms to see if the symptom is listed there. That may help you to identify the most likely root cause without having to read all of this section.

The common problems with certificate verification vary depending on who signed the visualization mod or the external action.

If a Spotfire user signs a mod or an external action then the certificate is issued by the Spotfire Server. We will discuss the common problems for this case in the Section Signed by a Spotfire user.

However, it is also possible to sign a mod using a certificate that is issued by a public certificate authority. These kinds of certificates are used when an organization develops a visualization mod that should be used in multiple Spotfire environments, possibly within other organizations. The visualization mods that Spotfire provides are signed with that kind of certificate. We will discuss the common problems for this case in Section Signed with a certificate issued by a public certificate authority.

A mod may also be signed using a certificate that is issued by a private certificate authority that is managed by your own organization. We will discuss that briefly in Section Signed with a certificate issued by a private certificate authority.

It may be tempting to try to solve issues using trial-and-error. However, you should be careful so that you do not introduce a security issue.

• There is no need to do anything in the Spotfire admin UI, such as adding a certificate to a group. That will never solve a certificate verification issue.
• You should not import a root certificate into the Windows certificate store. If you import a root certificate to the Windows certificate store as a trusted root certificate you may compromise the security of your computer.
• There is no need to import a certificate that was used to sign a mod into the Windows certificate store. It will never solve any issue. You should avoid it since you may accidentally import a root certificate which may compromise the security of your computer.
• There is usually no need to import an intermediate certificate into the Windows certificate store. You should avoid it since you may accidentally import a root certificate which may compromise the security of your computer. If you have a working internet connection then Windows will fetch any needed intermediate certificates automatically. If you do not have a working internet connection then it may have an effect to import an intermediate certificate, and you may get a different error message in Spotfire. However, Spotfire requires a working internet connection to perform the certificate revocation check anyway. There is one unusual exception to this rule when you may resolve the problem by importing an intermediate certificate. See Section Issue with caching of intermediate certificates. 

In general, there is no need to import any certificate into the Windows certificate store. So, to avoid the risk of making a mistake it is best to avoid it.

Finally, beware that a mod may be signed with a forged certificate by a user with malicious intent. Spotfire will and should fail to verify such certificates.

Signed by a Spotfire user

When a user creates a mod using the Spotfire client it will be signed with a certificate that belongs to the user. That certificate will be issued by the current Spotfire Server cluster.

A common problem is that Spotfire is not able to verify a certificate that is issued by another Spotfire Server. A common situation where this happens is if you have signed a mod in a test system and try to use it in your production system.

If you want your Spotfire system to be able to verify a mod that is signed by a user in another Spotfire system then you need to configure your system accordingly. See Section How-to: Configure Spotfire to accept signatures from other Spotfire systems.

You can see whether a mod, most likely, has been signed by a user in a Spotfire system if you look at the signature details.  

Here, the error message says that "The entity might be signed by a user in another Spotfire system". Spotfire cannot know for sure that the mod is signed by another Spotfire system. It may also be the case that the signature is forged.

You may also look at "Certificate issuer:". If the certificate issuer is of the form "CN=TIBCO Spotfire Code Trust Root CA" then the mod has been signed by a Spotfire user (unless the signature has been forged). The id "d5e76cda-47d2-49cf-abdd-65014c45cbb2c" is a unique identifier associated with the Spotfire server cluster that issued the certificate.

Revocation checks

When Spotfire verifies a certificate it checks with the issuer whether the certificate has been revoked, by issuing an HTTP request. when the certificate has been issued by another Spotfire system, your Spotfire system needs to be able to reach the other Spotfire system. If that is not possible the certificate verification will fail.

Thus, if you configure one of your Spotfire systems to accept signatures from another Spotfire system then your system will require that the other Spotfire system is available when it verifies a certificate issued by that system. This may cause problems if you maintain or retire the other system. Because of this, it is important to consider configuring the public address for your different environments. This would mean it may be possible to retire systems, as long as the revocation information still exists in the database. However, if your organization has multiple Spotfire systems you may consider signing the mods that should be used across the system using a certificate issued by a public certificate authority.

Mods signed by a user in an unknown Spotfire system

If a mod is signed by a user in another unknown Spotfire system then you should not configure your system to trust the unknown system. Also, If the mod is signed in a third-party Spotfire system, such as the system of a supplier, we do not recommend that you configure your system to trust the third-party Spotfire system.

If you want to use a mod that has been developed by someone outside of your organization you have a couple of options.

If you have purchased the right to use the mod from another organization you should ask them to sign the mod using a certificate that has been issued by a public certificate authority. This is the standard procedure for commercial software.

You may also ask the developer to provide the source code of the mod. This may be appropriate, for example, if the mod is developed by an open-source project. You may then import the mod project into Spotfire as if the mod was developed by yourself. The mod will then be signed by your Spotfire user account in your system.

Finally, if your organization has a certificate that is issued by a public certificate authority you may sign the mod using that certificate. To do that you use the Spotfire Package Builder console tool.

Signed with a certificate issued by a public certificate authority

A mod may be signed using a certificate that is issued by a public certificate authority.

These kinds of certificates are used when an organization signs a mod. The visualization mods that Spotfire provides are signed with that kind of certificate.

The role of the certificate authority is to verify the identity of an organization that purchases a certificate. If a person can prove that they legally represent an organization then the certificate authority will issue a certificate with the name of the organization together with a secret private key that the organization can use to sign software artifacts (such as visualization mods).

Put simply, verifying a public certificate requires that Spotfire verifies two things. First, Spotfire needs to verify that the certificate has been issued by a legitimate certificate authority. Next Spotfire needs to verify that the certificate has not been revoked.

The second step requires that Spotfire is able to send HTTP requests over the internet to a server managed by the certificate authority, to query whether a certificate is revoked. In some cases, the first step also requires that Spotfire can connect over HTTP to the certificate authority, in order to fetch intermediate certificates.

Lack of internet connection

By far, the most common problem is that Spotfire is not able to connect to the internet.

If a mod is used in the Spotfire Analyst client then the certificate verification takes place on the client machine. That means that the HTTP requests to the certificate authority are issued from the desktop machine.

However, if the mod is used in the Spotfire Web Player then the verification takes place in the worker services on the node manager machine. In that case, the HTTP requests to the certificate authority are issued from the node manager machine. It is common that servers do not have a direct internet connection.

If it works to verify the certificate in the desktop client but not in the Web Player then it strongly indicates that the node manager machine does not have internet access.

The Spotfire server also verifies certificates under some circumstances. The certificate verification on the Spotfire server can be disabled without compromising security. We describe that in Section How-to: Disable certificate validation on the Spotfire server.

Using a proxy server

If a server does not have direct access to the internet it is customary to use a proxy server. The proxy server needs to run on another machine that has direct access to the internet and it needs to be configured appropriately. How to configure a proxy server is beyond the scope of this article and it depends on which proxy server your organization uses. However, there are two things to keep in mind which we discuss here.

Proxy URL allow-list

Firstly, some proxies are configured using an allow-list which specifies which URLs the proxy will forward. If your proxy is configured to use an allow-list then the URLs to certificate authorities may be on the allow-list already. However, if they are not then you need to find the URLs by inspecting the certificate and adding those URLs to the allow-list. We discuss this in Section How-to: Find URLs that need to be forwarded by a proxy server.

Proxy Authentication

The second thing to keep in mind is that your proxy server may require authentication. If so, it matters which user that issues the requests to the proxy server. We discuss this in Section How-to: Find the user account(s) that is used to verify certificates.

There are two pitfalls here. Firstly, if the Local System account is used to run the Spotfire service then the Local System account needs to be able to access the proxy server. If your proxy server is configured to allow all users to connect to the proxy then it may not include the Local System account because it is not a proper user account. You can try to run the service with your own user account to see whether this solves the problem. If it does, we recommend that you use a dedicated service account to run the service.

Secondly, if the system is configured to use Kerberos, then the proxy server needs to be accessible to all users of Spotfire. Also, delegation in Kerberos needs to be configured properly in the domain controller. How to configure delegated Kerberos is outside the scope of this article. If you use Kerberos, then you may want to disable impersonation during certificate verification to avoid this pitfall. See Section How-to: Disable impersonation for Kerberos when verifying certificates.

To troubleshoot the proxy you can use the Windows certificate viewer. See Section How-to: Troubleshoot using Windows certificate viewer. In advanced cases you may also use the command certutil, which is available by default in Windows. See Section How-to: Troubleshoot using certutil.

Other Issues

By far, the most common problem is that Spotfire is not able to connect to the internet. See Lack of internet connection and Using a proxy server. However, we have seen two other issues that can occur even with an internet connection. We discuss those issues in this section.

Issue with caching of intermediate certificates when using Kerberos

When Spotfire is configured to use Kerberos then the Web Player service process impersonates the end user. This may cause issues if you use a proxy that requires authentication. See Section Proxy Authentication.

However, impersonation can also cause issues even if you have a direct connection to the internet. We believe that this is caused by access rights issues related to the caching of intermediate certificates in the registry. The problem seems to be that intermediate certificates are cached on behalf of the end user using the access rights of the process identity. If the process identity has insufficient access rights then the certificate verification will fail.

The issue can be resolved by disabling impersonation during certificate verification. See Section How-to: Disable impersonation for Kerberos when verifying certificates.

Issue with caching of intermediate certificates

Another reason why certificate verification may fail is that an intermediate certificate cannot be cached. This is a rare issue which we believe is because of insufficient access rights in the registry.

If you have an issue in the Analyst client on a machine that has internet access then it may be because of this issue. Note that in this case, the certificate viewer will not be able to verify the certificate either because it cannot build the certificate chain. See Section How-to: Troubleshoot using Windows certificate viewer.

If a visualization mod or external action is signed by a signing certificate that has been issued by an intermediate certificate authority, the intermediate certificate is needed to build the certificate chain to the trusted root. To verify the chain Spotfire uses the content of the Windows Certificate Store on the machine that runs the Analyst. If the intermediate certificate isn't present in the Windows Certificate Store, Spotfire tries to download the certificate (which is done with the help of the Authority Information Access information in the digital certificate). It does not just download the certificate, it also populates the Windows Certificate Store with this certificate, as the certificate chain will be built by using the content of the store.

If the person that runs the application does not have sufficient permissions in the registry to add certificates, the user might not be able to populate the Windows Certificate Store and the verification will fail because of an incomplete certificate chain.

This is a bit of a subtle issue, and there could be other issues playing a part. To investigate this more thoroughly, get a hold of the intermediate certificate by using the information in the Authority Information Access in the signing certificate, see Section How-to: Find URLs that need to be forwarded by a proxy server. When you have the intermediate signing certificate, you can check whether this certificate is present in the Windows Certificate Store on the machine running Spotfire. If you can not find it in the store, the issue points towards an issue with not being able to populate the store.

A good experiment would be to also double-click on the signing certificate and look at the certification path in the Windows certificate viewer. When this is done, just as when Spotfire tries to verify the certificate, should the Windows Certificate Store be populated with the possible intermediate certificate. If the certification path cannot be built, the issue would point toward not having sufficient permissions to populate the store.  

A solution to this issue would be to have an administrator import the intermediate certificate to the Local Computer Certificate Store, in the category Intermediate Certificate Authorities. This import should solve the issue, which means that if you again double-click on the certificate the certification path to the root certificate should be visible in the Windows certificate viewer.

Signed with a certificate issued by a private certificate authority

It is possible to sign mods with a certificate that has been issued by a private certificate authority. A private certificate authority is a certificate authority managed by your own organization. To manage a private certificate authority is non-trivial and a mistake may breach the security of your system, so you should not do that unless you know very well what you are doing.

If your private certificate authority uses an intermediate certificate that is issued by a public certificate authority then Section Signed with a certificate issued by a public certificate authority applies. However, the machine that verifies the certificate will also need to be able to issue HTTP requests to your private certificate authority.

If your private certificate authority has issued its own root certificate then you need to ensure that the root certificate is trusted when Spotfire verifies the signature of the mod.  See Section How-to: Distribute the root certificate of a private certificate authority

How-to: Troubleshoot using Windows certificate viewer

In this section, we describe some basic troubleshooting using the Windows certificate viewer. This only applies to certificates that have been issued by a public certificate authority, such as the certificate that Spotfire uses to sign visualization mods. See Signed with a certificate issued by a public certificate authority.

The first step is to download the certificate from the Spotfire client. See Section Common symptoms in the Clients.

You can then double-click on the certificate to open it in the Windows certificate viewer.

The screenshots above show the certificate that Spotfire currently uses for signing visualization mods at the time of writing this. (Certificates expire periodically so in the future Spotfire will use another certificate). Here the Certificate status says that the certificate is ok.

You can check whether a certificate is ok using any Windows machine that has a working internet connection. However, to troubleshoot it is important that you run the certificate viewer on the same machine where Spotfire runs.

• If you are troubleshooting the Analyst client then you need to run the certificate viewer on the same machine.
• If you are troubleshooting the Web Player client then you need to run the certificate viewer on the Node Manager machine that runs the Web Player service.
• If the symptom occurs in the admin UI (See Section Common symptom in the Admin UI), when executing a CLI command (See Section Common symptom in the CLI), or if it is a symptom in the client associated with the server (See Section Error when trusting and Slow to save a mod to the library) then you should run the certificate viewer on the Spotfire Server machine.

Spotfire verifies certificates using the same underlying Windows API as the certificate viewer. However, it may happen that Spotfire fails to verify a certificate even if the certificate viewer succeeds. This can indicate what the underlying issue is.

Analyst client machine

If the issue is in the Analyst client and the certificate viewer also fails to verify the certificate then that indicates that:

• There is no direct internet connection. In this case, you can use a proxy. See Section Using a proxy server.
• If there is an internet connection, then there may be an issue with the caching of intermediate certificates. See Section Issue with caching of intermediate certificates.

Web Player Node Manager machine

If the issue is in the Web Player client and the certificate viewer also fails to verify the certificate then that indicates that:

• There is no direct internet connection. In this case, you can use a proxy. See Section Using a proxy server.
• If there is an internet connection, then there may be an issue with caching of intermediate certificates. See Section Issue with caching of intermediate certificates.

If the issue is in the Web Player client and the certificate viewer succeeds in verifying the certificate then that indicates that:

• An issue with authentication towards a proxy. See Section Proxy Authentication.
• An issue with caching of intermediate certificates due to impersonation when using Kerberos. See Section Issue with caching of intermediate certificates when using Kerberos.

Spotfire Server machine

If the issue is in the Server and the certificate viewer also fails to verify the certificate then that indicates that:

• There is no direct internet connection. In this case, you can use a proxy. See Section Using a proxy server.

If the issue is in the Server and the certificate viewer succeeds in verifying the certificate then that indicates that:

• An issue with authentication towards the proxy. See Section Proxy Authentication.

How-to: Collect information when reporting a support case

In this section, we list information that will help Spotfire to troubleshoot a certificate verification issue. It will help to speed up a support case if you provide this information when you open the case.

Always include:

• Which issue(s) listed in Section Common symptoms have you observed? Have you observed an issue that is not listed in that section which may be related to certificate verification? Make screenshots of the error message shown in the client.
• The certificate that is used to sign the mod in the support case. See Section Common symptoms in the Clients on how to get the certificate.
• If it is a symptom in the client, does it happen in the Analyst client, the Web Player, or both?

What else is useful depends on the kind of issue you are reporting.

If the symptom occurs in the admin UI (See Section Common symptom in the Admin UI), when executing a CLI command (See Section Common symptom in the CLI), or if it is a symptom in the client associated with the server (See Section Error when trusting and Slow to save a mod to the library) then this is most likely an issue with the certificate verification on the Spotfire server machine. See Section Spotfire Server machine.

If it is another symptom in the Web Player client then it is most likely an issue with the certificate verification on the Node Manager machine. See Section Web Player Node Manager machine.

If the symptom is in the Analyst client then See Section Analyst client machine.

Spotfire Server machine

If it is an issue related to certificate verification on the Spotfire server machine then include:

• Is the issue resolved if you disable the certificate verification in the Spotfire Server? See Section How-to: Disable certificate validation on the Spotfire server.
• A screenshot when you open the certificate in the Windows Certificate viewer on the Spotfire server. See Section How-to: Troubleshoot using Windows certificate viewer.
• A troubleshooting bundle that includes the Server logs.
• Do you have direct internet access from the Spotfire Server machine or do you use a proxy?
• If you use a proxy,

• Which account is running the Spotfire Server service? See Section How-to: Check which account runs a service.
• The output of certutil when you run it on the Spotfire server. See Section How-to: Troubleshoot using certutil.

Web Player Node Manager machine

If it is an issue related to certificate verification on the Node Manager machine that runs the Web Player service then include:

• A screenshot when you open the certificate in the Windows certificate viewer on the Node Manager machine that runs the web player service. See Section How-to: Troubleshoot using Windows certificate viewer.
• A troubleshooting bundle that includes the Server logs and the Web Player logs at the DEBUG level. Make sure to enable the logging on the DEBUG level first and then reproduce the issue before you capture the logs.
• Have you configured Spotfire to use Kerberos? See Section How-to: Check if Spotfire is configured to use Kerberos.
• If you have configured Spotfire to use Kerberos, is the issue resolved if impersonation is disabled during certificate verification? See Section How-to: Disable impersonation for Kerberos when verifying certificates.
• Do you have direct internet access from the Node Manager machine or do you use a proxy?
• If you use a proxy,

• Which account is running the Node Manager service? See Section How-to: Check which account runs a service.
• The output of certutil when you run it on the Node Manager machine. See Section How-to: Troubleshoot using certutil.

Analyst client machine

If it is an issue related to certificate verification on the Analyst client machine then include:

• A screenshot when you open the certificate in the Windows certificate viewer on the machine that runs the Analyst client. See Section How-to: Troubleshoot using Windows certificate viewer.
• The Analyst logs at the DEBUG level. You can enable logging using "Help - Diagnostic and Logging". Make sure to enable the logging on the DEBUG level first and then reproduce the issue before you capture the logs.

How-to: Disable certificate validation on the Spotfire server

If your Spotfire Server does not have internet access it will not be able to verify certificates. You may use a proxy server to solve that issue. See Section Using a proxy server. However, the certificate verification on the Spotfire server can be disabled without compromising security. In this section, we will describe how to disable the certificate verification in the Spotfire server.

The verification of certificates on the server happens when

• a user trusts a mod (see Section Error when trusting),
• when a mod is saved in a library (see Section Slow to save a mod to the library),
• when a certificate is imported in the admin UI (see Section Admin UI) and
• when a certificate is imported using a CLI command (see Section CLI).

To disable the verification of certificates when making a trust decision, uploading a certificate through the command line or Admin UI, the configuration property security.code-trust.validate-uploaded-cert must be set to false. This can be done through the following steps:

1. On the Spotfire Server machine, open an Administrators command prompt and move to <TSS Installation Directory>/tomcat/spotfire-bin
2. Export current configuration through the command

• (Windows) ./config.bat export-config -t TOOL_PASSWORD
• (Linux) ./config.sh export-config -t TOOL_PASSWORD

3. Set the configuration property through the command:

• (Windows) ./config.bat set-config-prop -n "security.code-trust.validate-uploaded-cert" -v "false"
• (Linux) ./config.sh set-config-prop -n "security.code-trust.validate-uploaded-cert" -v "false"

4. Import the updated configuration to the database through the command:

• (Windows) ./config.bat import-config -c "comment your change"  -t TOOL_PASSWORD
• (Linux) ./config.sh import-config -c "comment your change"  -t TOOL_PASSWORD

5. Restart the server.

To turn off validation of the signature when saving a .mod-file to the library, the configuration property library.verify-code-trust-signature must be set to false. This is done in the same way as described above, but with the property name being changed in step 3.

To reiterate, there are no negative security implications of turning off server-side verification - this verification is just there for early error detection.

How-to: Configure Spotfire to accept signatures from other Spotfire systems

Spotfire can verify signatures from another Spotfire system if an administrator configures your Spotfire system to accept signatures from the other Spotfire system. Note that you should not configure your Spotfire system to accept signatures from an unknown Spotfire system. However, you may, for example, want to configure your test and production system to accept each other's signatures.

To configure your Spotfire system to trust another Spotfire system you need access to the root certificate of the other system. To export the root certificate you need access to the other system.

Your Spotfire system also needs to be able to send HTTP requests to the other system. We discuss that further in Section Revocation checks.

Export a root certificate from a Spotfire Server

To export the root certificate you log in to a Spotfire Server machine and use the administrative console. Write the following command.

1. Move into <TSS Installation Directory>/tomcat/spotfire-bin
2. Export the certificate through the command:

• (Windows) ./config.bat export-code-signing-root-certificate -t TOOL_PASSWORD -d TARGET_DIRECTORY
• (Linux) ./config.sh export-code-signing-root-certificate -t TOOL_PASSWORD -d TARGET_DIRECTORY

This will export the root certificate of the current server to TARGET_DIRECTORY/root-ca.cer.

Import a root certificate into a Spotfire Server

To import this certificate to the new environment, move this certificate to the machine that hosts that particular server. Open an administrative console.

1. Move into <TSS Installation Directory>/tomcat/spotfire-bin
2. Import the certificate through the command:

• (Windows) ./config.bat import-code-signing-certificate -t TOOL_PASSWORD -p PATH_TO_CERTIFICATE
• (Linux) ./config.sh import-code-signing-certificate -t TOOL_PASSWORD -p PATH_TO_CERTIFICATE

This will import the root certificate to the database. In order for this to take effect, the Analyst clients and Web Player services need to be restarted. The server verification will take the imported certificates into account at once, without any restart of the server. 

How-to: Check if Spotfire is configured to use Kerberos

To check if the Spotfire server is configured to use Kerberos authentication look in the server configuration, which can be found in the troubleshooting bundle, under security and auth-method.

How-to: Check which account runs a service

To check which account the services are running as, open Services and, for the given service, look under Log On As.

How-to: Find the user account(s) that is used to verify certificates

When troubleshooting certificate verification it may be important to know which account is used when Spotfire verifies a certificate.

In particular, this is important if you use a proxy server to provide internet access for the machine that runs the Spotfire Node Manager or the Spotfire Server. See Section Using a proxy server. It is common that proxy servers require authentication and the account needs to be able to authenticate with the proxy server.

Spotfire Server

The account that is used to run the Spotfire Server service is used when the Spotfire Server verifies a certificate. See How-to: Check which account runs a service.

Web player

In most cases, the account that is used to run the Node Manager service is used when the Web Player verifies a certificate. See How-to: Check which account runs a service.

However, if the system is configured to use Kerberos then that is not the case. If you don't know whether your system is configured to use Kerberos then see How-to: Check if Spotfire is configured to use Kerberos.

If the system is configured to use Kerberos then the account that is used to run the Node Manager will impersonate the end user. This has the effect that all HTTP requests are issued on behalf of the end user. This includes the HTTP requests that are issued while Spotfire verifies a certificate.

Thus, if the system is configured to use Kerberos, then a proxy server needs to be accessible to all users of Spotfire. Also, delegation in Kerberos needs to be configured properly in the domain controller. How to configure delegated Kerberos is outside the scope of this article.

It is possible to disable impersonation when a certificate is verified. See Section How-to: Disable impersonation for Kerberos when verifying certificates. If so, then the account that runs the service is used to verify the certificate. See How-to: Check which account runs a service.

How-to: Disable impersonation for Kerberos when verifying certificates

When Spotfire is configured to use Kerberos then the Web Player service process impersonates the end user. This means that http(s) requests will be made on behalf of the end user. The primary purpose is to ensure that data sources are accessed on behalf of the end user with a single sign-on experience.

This can cause issues for certificate verification for two reasons.

Firstly, if you use a proxy to connect to the internet and the proxy requires authentication then it will be the end user that needs to authenticate. This requires that the end-user has access to the proxy and that Kerberos is configured so that the credentials may be delegated. See Section Using a proxy server for discussions on how to use a proxy server.

Secondly, the impersonation can also cause issues with access rights in the registry. See Section Kerberos impersonation registry access rights.

It is possible to avoid these issues by forcing Spotfire to use the Web Player service process identity when verifying a certificate. If Spotfire is configured to use Kerberos then this will temporarily disable the impersonation during certificate verification.

The behavior is controlled by the setting VerifyCertificateUsingProcessIdentity in Spotfire.Dxp.Worker.Host.exe.config. By default the value is false, and you need to set it to true to disable impersonation.

To change the setting you need to export the config file using export-service-config. When you have changed the value you need to import the modified config file using import-service-config.

After having imported the modified config file, the config needs to be set as active as well. This can either be done by specifying the flag --apply when running import-service-config or with the set-active-config command.

Note that if you want to try the change first, before changing the configuration in the database, you can make the change in the configuration Spotfire.Dxp.Worker.Host.exe.config located in the <TSNM installation directory>/nm/services/<current service> directory.

How-to: Find URLs that need to be forwarded by a proxy server

If you use a proxy server to provide internet access for the machine that runs the Spotfire Node Manager or the Spotfire Server then you may need to add specific URLs to an allow-list in the proxy server in order for the proxy server to forward those URLs.

You can find the URLs that the proxy server needs to forward by inspecting the certificate that was used to sign the mod. You can download the certificate using the 'manage trust' dialog in the Spotfire client or using the Admin UI.

When you have downloaded the certificate you can open it in the Windows certificate viewer (on a machine with an internet connection), by double-clicking on the certificate. Then go to the Details tab and select Authority Information Access.

You will see two URLs that you should add to the allow-list of the proxy. The URL listed under Access Method=On-line Certificate Status Protocol is used to check whether the certificate is revoked. This is always necessary. The URL listed under Access Method=Certification Authority Issuer is used to fetch the certificate that issued this certificate. That may not be needed if that certificate is already in the certificate store.

The next step is to go to the Certification Path tab.

First check the below Certificate status: that the certificate status is "This certificate is OK". If that is not the case then you are probably running the certificate viewer on a machine that does not have an internet connection.

If the certificate is OK, then the Certification path contains all certificates that are needed to verify the certificate. The certificate that is at the bottom of the list is the certificate that you are viewing in the Certificate Viewer. The certificate that is at the top of the list is called the root certificate.

The other certificates are known as intermediate certificates. In the screenshot above "Symantec Class 3 SHA 256 Code Signing CA" is an intermediate certificate.

It is often the case that there are no intermediate certificates. Then you are done. However, if there are any intermediate certificates then you need to extract the URLs from the intermediate certificates. To do that you select the certificate in the Certification path: and click on View Certificate. You will then launch another instance of the Certificate Viewer which displays the intermediate certificate. The URLs of interest are found in the same way as for the code signing certificate, under Authority Information Access on the Details tab.

How-to: Troubleshoot using certutil

To troubleshoot certificate verification you can use the command certutil, which is available by default in Windows, to verify the certificate that was used to sign the mod. Spotfire verifies certificates using the same underlying Windows API as certutil uses.

Running Certutil

You should run certutil as the user account that Spotfire uses to verify certificates. This is important if you use a proxy server that requires authentication. See Section How-to: Find the user account(s) that is used to verify certificates.

Running certutil on the Node Manager machine when using Kerberos

If the system is configured to use Kerberos then the Web player service will be impersonated as the end user account that uses a mod when the certificate is verified. To test a specific user you should log in to the Node Manager/Spotfire Server machine as that user and open a Command Prompt and type the following command.

 certutil -verify /path/to/certificate

If certutil can verify the certificate but Spotfire fails then it may be an issue with the configuration of Kerberos in the domain controller.

Note that this does not apply if the impersonation has been disabled. See Section How-to: Disable impersonation for Kerberos when verifying certificates.

Running certutil when using Local System

If the service is running using the Local System account then you should login to the Node Manager/Spotfire Server machine. Open the Command Prompt using Run as administrator and type the following command.

 certutil -verify /path/to/certificate -sid 22

The switch -sid 22 tells certutil to verify the certificate using the Local System account. If the previous command fails to verify the certificate you can try the following command which will verify the certificate with the current user account.

 certutil -verify /path/to/certificate

If that command can verify the certificate then you most likely have a proxy that requires authentication and does not allow Local System to connect. If so you need to use a service account to run the service (unless you are using Kerberos, see the previous section).

Running certutil when using a Service account

If the service is running using a service account then you should login to the Node Manager/Spotfire Server machine using that account. Open the Command Prompt and type the following command.

 certutil -verify /path/to/certificate

Certutil Output

The certutil tool produces a lot of detailed output which may be hard to understand. Note in particular that the message "CertUtil: -verify command completed successfully." does not mean that certutil has managed to verify the certificate. It just means that command could execute.

Incomplete certificate chain

If certutil is unable to connect through the proxy then certutil may fail with the message "Incomplete certificate chain" as follows.

------------------------------------

Incomplete certificate chain

Cannot find certificate:

    CN=DigiCert SHA2 Assured ID Code Signing CA, OU=www.digicert.com, O=DigiCert Inc, C=US

Cert is an End Entity certificate

ERROR: Verifying leaf certificate revocation status returned The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613 CRYPT_E_REVOCATION_OFFLINE)

CertUtil: The revocation function was unable to check revocation because the revocation server was offline.

CertUtil: -verify command completed successfully.

Revocation check skipped -- server offline

If certutil is unable to connect through the proxy then certutil may also fail with the message Revocation check skipped -- server offline as follows.

------------------------------------

Revocation check skipped -- server offline

Cert is an End Entity certificate

ERROR: Verifying leaf certificate revocation status returned The revocation function was unable to check revocation because the revocation server was offline. 0x80092013 (-2146885613 CRYPT_E_REVOCATION_OFFLINE)

CertUtil: The revocation function was unable to check revocation because the revocation server was offline.

CertUtil: -verify command completed successfully.

Successful verification

Here is an example of the output when certutil succeeds to verify a certificate.

------------------------------------

Verified Issuance Policies: None

Verified Application Policies:

    1.3.6.1.5.5.7.3.3 Code Signing

Cert is an End Entity certificate

Leaf certificate revocation check passed

CertUtil: -verify command completed successfully.

How-to: Distribute the root certificate of a private certificate authority

If you use a private certificate authority that has issued its own root certificate then you need to ensure that the root certificate is trusted when Spotfire verifies the signature of the mod.

Warning: It is possible to import the root certificate to the Windows certificate store as a trusted root certificate. However, we advise against that, especially if the purpose of your private certificate authority is just to issue certificates for signing Spotfire mods. If you import a root certificate to the Windows certificate store as a trusted root certificate you may compromise the security of your computer.

Instead, you should import the root certificate of your private certificate authority to the Spotfire Server in the same way as you import the root certificate from another Spotfire system. See Section Import a root certificate into a Spotfire server.

You also need to configure Spotfire so that it distributes the root certificate using the setting security.code-trust.distribute-third-party-root-certs to true. This is done through the following steps

1. On the Spotfire Server machine, open an Administrators command prompt and move to <TSS Installation Directory>/tomcat/spotfire-bin
2. Export current configuration through the command:

• (Windows) ./config.bat export-config -t TOOL_PASSWORD
• (Linux) ./config.sh export-config -t TOOL_PASSWORD

3. Set the configuration property through the command:

• ./config.bat set-config-prop -n "security.code-trust.distribute-third-party-root-certs" -v "true"
• (Linux) ./config.sh set-config-prop -n "security.code-trust.distribute-third-party-root-certs" -v "true"

4. Import the updated configuration to the database through the command:

• (Windows) ./config.bat import-config -c "comment your change"  -t TOOL_PASSWORD
• (Linux) ./config.sh import-config -c "comment your change"  -t TOOL_PASSWORD

5. Restart the server.

Note that the Spotfire Server cannot distribute intermediate certificates in this way. Also, note that you should not distribute a root certificate from a public certificate authority in this way. Furthermore, if you have imported a root certificate in this way you can remove it by using the remove-code-signing-root-certificate command.