FAQ: Data function trust in Spotfire 10.3 and later

This article contains answers to frequently asked questions regarding changes to how scripts and data functions are trusted by authorized users in Spotfire release 10.3 and later.

Introduction

It is intended for Spotfire Administrators that plan to upgrade to Spotfire 10.3, and other users that want to understand more about the trust mechanism used to ensure scripts and data functions are safe to execute. For more information about the trust mechanism for client users, see the Spotfire Analyst User's Guide.

Spotfire 10.3 improved the trust mechanism for JavaScript and IronPython scripts, as well as for data connection custom queries, by introducing SHA-512 for calculating the trust stamp, and the trust mechanism now also applies to data functions. This means that there are a few additional steps required during the upgrade to Spotfire 10.3 and later, to make everything work as before. The exact steps to take will depend on your use case and environment and some examples are described below. In addition to the changes in script trust, a server command line interface (CLI) command, find-analysis-scripts, has been added to help with finding and analyzing scripts in existing analyses during an upgrade.

FAQ

Is the accessed time of the file analyzed changed when running the find-analysis-scripts command?

No.

If something goes wrong while running the find-analytics-scripts command, could this corrupt the analysis files being analyzed?

No, since it does not write the file to the library, it just writes the calculated trust stamp back to the library.

When the find-analysis-scripts tool downloads DXP files to analyze them, will it also load linked data, run information links, etc.?

No. It just downloads the DXP file and analyzes it for scripts.

Embedded data will be downloaded with the DXP file.

How do I limit the files from the Library I want to analyze? I want to analyze files on a specific folder or only files accessed the last month.

You can run find-analysis-scripts on a specific folder in the Library and/or use Library search expressions to limit the files. Please see more examples in this article.

When I try to run the find-analysis-scripts command it states that there are 0 non-analyzed scripts in the selected library folder. Why is this?

This is most likely because you already have run the find-analysis-scripts on the same library folder(s) and that no file has been modified since then.

If you want to trust scripts, you can use the generated file "trust_<date>.script".

Alternatively you can use the find-analysis-scripts command with the "--resume=false" parameter. This will force the tool to run on all files in the selected library folder(s).

When viewing the report that find-analysis-scripts generates, how do I see if a script or data function is trusted?

Look at the "Trusted" and "Automatically Trusted" columns in the report. The "Trusted" column indicates if the script or data function was already trusted before the run of find-analysis-scripts. The "Automatically Trusted" column indicates if the data function was trusted as a result of this run of the find-analysis-scripts.

When find-analysis-scripts command is running, where does it store the files it downloads?

The files are stored in the location indicated by the java configuration property java.io.tmpdir. The configuration property is set in config.bat on windows systems (config.sh on Linux). Note that downloaded files are deleted as soon as they have been analyzed.

How does the find-analysis-scripts command know what files it already analyzed?

The command stores it?s progress in a file called progress.xml, by default located in the [TIBCO Spotfire Server installation directory]\tomcat\spotfire-bin\find-analysis-scripts directory. If the progress.xml file is deleted the command will process everything in the path. The same effect is obtained by the option "--resume=false".

The find-analysis scripts command creates files such as progress.xml, report_[DATE]_.csv, trust_[DATE].script in the folder find-analysis-scripts (or another specified folder). When are they removed?

They are not automatically removed by Spotfire. They can however be removed manually or by automation should it be needed.

Does the find-analysis-scripts and trust commands log to any log files?

Yes, if there are issues, those are logged in the standard server configuration command log - [TIBCO Spotfire Server installation directory]\tomcat\logs\tools.log

From the manual:

tools.log

Information about activity in the configuration tool and on the command line. If you run any configuration commands at the command prompt or use the administration console, this is the log that captures that information.

There should in general be little need to tweak log level. It is however controlled by [TIBCO Spotfire Server installation directory]\tomcat\spotfire-config\log4j2-tools.xml

How do we configure the location for the analysis files to be downloaded by the find-analysis-scripts command?

Either set java.io.tmpdir (in config.bat/config.sh) or set the "attachment.temp-folder" configuration property (the latter would also affect running servers). Note that these files are only stored temporarily and are removed as soon as possible. Each analysis thread will at most analyze one file at a time - and it's possible to make the process single threaded by specifying "--single-threaded" (this will increase the execution time but will reduce the amount of disk space, memory, CPU and network bandwith used).

If a user would open a DXP file and change it while find-analysis-scripts is running and trusting data functions, what happens?

It depends on what the user changes. If the user changes a data function and saves it, that data function will not become trusted since it was changed. Nothing else happens, and if you run find-analysis-scripts again it will show that this file is non-analyzed (since it was modified after find-analysis-scripts ran).

If the user just does other changes to the DXP file, not modifying the data function, then the data function will be trusted.

If I run find-analysis-scripts, then I delete all the contents of the folder find-analysis-scripts before again running find-analysis-scripts, it does show that all files are non-analyzed. Is this an expected behavior?

Yes, that's the expected behavior (the same behavior can be achieved by specifying "--resume=false").

How do I know if a script that was trusted since before have been given a new SHA-512 based checksum or still works with the old SHA-1 checksum?

If the column called "Hash (SHA-512)" in the report generated by find-analysis-scripts has a value, then the scripts trust checksum has been updated. find-analysis-scripts does this automatically for all IronPython and JavaScript based scripts, as well as for data connection custom queries.

Spotfire 10.3 introduces the trust mechanism for data functions. But what about Expression functions and inline TERR expressions?

Expression functions can also be trusted. Inline TERR scripts cannot be trusted but are instead limited to using the Restricted execution mode of TERR. Read more in the main article about data function trust: https://community.spotfire.com/s/article/Script-and-Data-function-trust-in-Spotfire-10-3-and-later

I have noticed that when I trust a script or data function in one DXP file on the library, if the same script/data function is used in other DXP files I use, it seems it is automatically trusted there as well. Is this true?

Yes, the design of the trust feature means that if the same script/data function exists in several files you use in the library, trusting the script/data function in one of those implies it is trusted also in the other files

Sign In