Calling the Spotfire® UpdateAnalysisService Web Service

Scheduled Updates using the UpdateAnalysisService Web Service

To recap - this web service can be used to update an analysis on the Spotfire Web server by an external system. This first example will use .Net. We can update any analysis with or without an ScheduleUpdate rule. When you update an analysis without an ScheduleUpdate rule, the KeepAliveMinutes parameter value can be set to unload the analysis accordingly.   Example C# and Java code for Spotfire 7.13 or later is attached to this article.

Additional configuration for Spotfire Server 7.13 and more recent versions

Starting in Spotfire Server 7.13, a new OAuth 2.0-based version of the Web Services API was introduced.  The legacy version was removed in Spotfire 10.3.  To use the OAuth 2.0 protocol based authentication, one needs to register a client with the Spotfire Server and use the provided Client ID and Client secret to authenticate and obtain an authorization token.  

One needs to register the API client and scope first in order to get the client ID and client secret information from the Spotfire Server. 

1. On a Spotfire Server machine, open a command line window.
2. Change directory into tomcat/bin (before 10.3.0) or tomcat/spotfire-bin (after 10.3.0)
3. Run the command to configure the API client:

config register-api-client -n TestUpdateAnalysis -Sapi.soap.update-analysis-service
Tool password:
Successfully registered a new API client with the display name 'TestUpdateAnalysis':
Client ID: <YOUR CLIENT ID>
Client secret: <YOUR CLIENT SECRET>
To view the full client configuration, please use the 'show-oauth2-client' command.

Record the Client ID and Client Secret as these will be needed in your code to contact the Web Service API.  In the command example above, the name of the user is the "-n" parameter in this case "TestUpdateAnalysis," and access to the update analysis scope, api.soap.update-analysis-service, is requested.

Using Visual Studio .Net (C#)

1. Create a new Visual Studio Project and add a Web Reference

Create a new Visual Studio project and right click on the References section in Solution Explorer. Click "Add Service Reference..."

Now it gets slightly non-intuitive. WCF is Windows Communication Foundation - it's supposed to make life easier for developers to develop and consume web services, but in this case it doesn't play nicely with Spotfire, so we need to create a .Net 2.0 Web Reference instead of a Service Reference, so enter a namespace in the dialog, and click the Advanced... button to get to this dialog:

Click "Add Web Reference..."

Now fill in the URL to your web service depending on the version you are running:

• For Spotfire Server 7.12 and older: http(s)://spotfireserver:port/spotfire/ws/pub/UpdateAnalysisService?wsdl
• For Spotfire Server 7.13 and newer:  http(s)://spotfireserver:port/spotfire/api/soap/UpdateAnalysisService/wsdl

Click the right-arrow to connect to the Spotfire server and download the WSDL that defines the web service. At this point, you'll be prompted to enter username and password credentials. Annoyingly, this prompt often happens multiple times!

Enter a name for the Web reference. You should end up with something that looks like:

So go ahead and click "Add Reference".

2. Add the Necessary References to your Visual Studio Project

You will need references to System.ServiceModel and System.Web.Services, so add those in the usual way. On my installation they are in my recently used list:

Now you're ready to code!

3. Code the Web Service Client

In Spotfire 7.13, the authentication method for the Spotfire Server Web Services changed to using the OAuth2.0 protocol.  As noted earlier, one needs to register the API client to obtain the Client ID and Client Secret for authentication using the OAuth 2.0 protocol.  The following assumes one has already obtained the Client ID and Client Secret.

The following discusses changes to the authentication code after steps 1 and 2.  The first one is to create a class that will add the Bearer authorization header to the requests made to the Web Service.  This requires extending System.Web.Services.Protocols.SoapHttpClientProtocol:

/// <summary>Custom implementation of soap proxy to create OAuth2 requests.</summary>
public class MySoapHttpClientProtocol : System.Web.Services.Protocols.SoapHttpClientProtocol
{
	private String _OAuthBearerToken = "";

	public String OAuthBearerToken
	{
		get { return _OAuthBearerToken; }
		set { _OAuthBearerToken = value; }
	}

	protected override System.Net.WebRequest GetWebRequest(Uri uri)
	{
		var request = base.GetWebRequest(uri);
		request.Headers.Add("Authorization", "Bearer " + OAuthBearerToken); // <----
		return request;
	}
}

In order for this extended class to be called, one must change the code that was auto-generated when referencing the Web Service:

..............
/// <remarks/>
[System.CodeDom.Compiler.GeneratedCodeAttribute("System.Web.Services", "4.7.2556.0")]
[System.Diagnostics.DebuggerStepThroughAttribute()]
[System.ComponentModel.DesignerCategoryAttribute("code")]
[System.Web.Services.WebServiceBindingAttribute(Name="UpdateAnalysisServiceImplServiceSoapBinding", Namespace="http://spotfire.tibco.com/ws/2015/08/externalScheduledUpdate.xsd"")]
public partial class UpdateAnalysisServiceImplService : MySoapHttpClientProtocol //<-----------
{
        
    private System.Threading.SendOrPostCallback loadAnalysisOperationCompleted;
...................

After that is done, one can modify the original code as follows for the modified authentication.

using System.ServiceModel.Description;
using SpotfireWebServices.updateAnalysis;
using System.IO;
using System.Web.Script.Serialization;

namespace MyTestProgram
{
    class Program
    {
        static void Main(string[] args)
        {
            // Create a reference to the UpdateAnalysisService
            UpdateAnalysisServiceImplService test = new UpdateAnalysisServiceImplService();
			/*
			 * Get OAuth Token for API
			 *
			*/
           // config register-api-client --name TestUpdateAnalysis -Sapi.soap.update-analysis-service
            /*
            config register-api-client -n TestUpdateAnalysis -Sapi.soap.update-analysis-service
            Tool password:
            Successfully registered a new API client with the display name 'TestUpdateAnalysis':
            Client ID: <YOUR CLIENT ID>
            Client secret: <YOUR CLIENT SECRET>
            To view the full client configuration, please use the 'show-oauth2-client' command.
            */

            string oAuthClientID = <CLIENTID>;
            string oAuthClientSecret = <CLIENTSECRET>;
            string urlOAuth = "http://spotfireserver:port/spotfire/oauth2/token"";

            try
            {
                System.Console.WriteLine("Retrieving OAuth Token from: " + urlOAuth);

                HttpWebRequest wrOAuth = (HttpWebRequest)WebRequest.Create(urlOAuth);

                string oAuthClientInfo = WebUtility.UrlEncode(oAuthClientID) + ":" + WebUtility.UrlEncode(oAuthClientSecret);
                string base64OAuth = Convert.ToBase64String(Encoding.UTF8.GetBytes(oAuthClientInfo));
                wrOAuth.Headers["Authorization"] = "Basic " + base64OAuth;
                wrOAuth.Method = "POST";
                wrOAuth.ContentType = "application/x-www-form-urlencoded;charset=UTF-8";
                byte[] postBytes = Encoding.UTF8.GetBytes("grant_type=client_credentials&scope=" + WebUtility.UrlEncode("api.soap.update-analysis-service")); 
                wrOAuth.ContentLength = (long)postBytes.Length;
                using (Stream requestStream = wrOAuth.GetRequestStream())
                {
                    requestStream.Write(postBytes, 0, postBytes.Length);
                }

                HttpWebResponse response = (HttpWebResponse)wrOAuth.GetResponse();

                Stream responseStream = response.GetResponseStream();
                if (responseStream == null)
                {
                    throw new WebException("Response contained no data.");
                }
                // parse out JSON return data
                JavaScriptSerializer javaScriptSerializer = new JavaScriptSerializer();
                string accessToken = null;
                using (StreamReader streamReader = new StreamReader(responseStream, Encoding.UTF8))
                {
                    string input = streamReader.ReadToEnd();
                    Dictionary<string, object> dictionary = (Dictionary<string, object>)javaScriptSerializer.DeserializeObject(input);
                    foreach (KeyValuePair<string, object> current in dictionary)
                    {
                        string key;
                        if ((key = current.Key) != null)
                        {
                            if (key.Equals("access_token", StringComparison.OrdinalIgnoreCase))
                            {
                                accessToken = (string)current.Value;
                                continue;
                            }
                            if (key.Equals("token_type", StringComparison.OrdinalIgnoreCase))
                            {
                                continue;
                            }
                            if (key.Equals("expires_in", StringComparison.OrdinalIgnoreCase))
                            {
                                continue;
                            }
                        }
                    }
                }
                test.OAuthBearerToken = accessToken;
				
            }
            catch (Exception ex)
            {
                System.Console.WriteLine("Exception calling OAuth Token URL, " + urlOAuth + ": " + ex.Message);
                throw ex;
            }

            // Create a new UpdateAnalysis object and set its path. There are more properties that can be set
            // but in this case we only need the path
            loadAnalysisUpdateAnalysis analysisToUpdate = new loadAnalysisUpdateAnalysis();
            analysisToUpdate.path = "/My Analysis to Update";

            // Call the web service to load the analysis
            test.loadAnalysis(analysisToUpdate);
        }
    }
}

The code calls the Spotfire Server OAuth2 endpoint with the client ID and client secret to obtain the bearer token which is then used in the authorization header when calling the Web Services API endpoint.  Obviously your namespaces and Using directives may be slightly different, depending on what you entered.

Testing and Configuring

This client should just work when run! If you get 403 errors, this may mean that the web services API might not be enabled or that your user does not have the proper permissions.  For Spotfire Server 7.12 and older, the client user must be a member of the built-in "API User" group. For all versions, the user needs to have the "External updates of analysis files in Spotfire web clients" license via one of its groups.  This screen shot shows the WPUpdate user group with only this license enabled:

Please check these things!

You can configure the URL of the web service by editing the Settings.settings in your project - it should be created automatically for you:

Password Encryption

Finally, a word about encryption and password storage. It is likely that this will form part of a scheduled application and so you'd like to store the username and password credentials somewhere. It's beyond the scope of this article to go into that in full detail, but a suggestion could be to use the Microsoft Cryptography classes in System.Security.Cryptography.

See here for more information: How to: Use Data Protection (External Microsoft MSDN link)

Using Java

Using Java to call the web service is a little more straightforward. In this example, Eclipse is being used, but any Java IDE can be used, or it can be coded in Notepad or similar.

1. Create an Empty Project and a Main Class

Create an empty project and add a class with a main method. This is done in Eclipse by clicking New->Class... within the project. Click "public static void main..." under "Which method stubs would you like to create?":

2. Generate the Method and Object Proxies for the Web Service

Navigate to the root directory of your project in a command prompt. For example:

C:\users\myuser\workspace\webservicesproject

For Spotfire Server 7.12 and older, create an auth file (call it auth.txt) for the web service - it must contain a line like this:

 http(s)://username:password@spotfireserver:port/spotfire/ws/pub/UpdateAnalysisService?wsdl

Run the wsimport command to generate the proxies:

 wsimport -d bin -s src -Xauthfile auth.txt http(s)://spotfireserver:port/spotfire/ws/pub/UpdateAnalysisService?wsdl

After JDK 8, the wsimport tool is not included in the JDK.   A wsimport tool is available in the JAX-WS open source project.  I used version 2.3.2 from this link in August 2019: https://search.maven.org/search?q=g:com.sun.xml.ws%20AND%20a:jaxws-ri&core=gav.  I downloaded the files and unzipped the downloaded zip file.  In the resulting bin directory, one will find a wsimport.bat and wsimport.sh file.  I used this example with JDK 11.  Prior to running the wsimport.bat/sh one will need to set the JAVA_HOME variable if not already set.  Open a command line window and change directory to the jax-ws/bin directory.  The parameters are the same as the wsimport tool:

 wsimport.bat -d bin -s src http(s)://spotfireserver:port/spotfire/api/soap/UpdateAnalysisService/wsdl

Once that is done, go back to Eclipse (or your Java IDE) and refresh the project. You should see a layout similar to this:

3. Write the Code to Call the Web Service

The code for calling the web service for Spotfire Server 7.13 and later is a bit more complicated.  The authentication mechanism has changed to use the OAuth 2.0 protocol. This changes the authentication code in the example code above.  The call to the Web Service is still the same and requires Steps 1 and 2.  As discussed earlier, one needs to register the API client and scope first in order to get the client ID and client secret information from the Spotfire Server. 

The following assumes one has already registered the API client and has the client ID and client secret.  First, we create a class that will add the Bearer authorization header to the requests made to the Web Service:

package com.tibco.spotfire.webservices.UpdateAnalyisService;

import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.xml.ws.handler.LogicalHandler;
import javax.xml.ws.handler.LogicalMessageContext;
import javax.xml.ws.handler.MessageContext;

public class TSSWSAPIHandler implements LogicalHandler<LogicalMessageContext> 
{
	private List<String> oauthBearerToken = null;
	
    public boolean handleMessage(LogicalMessageContext context) 
    {
        return processMessage(context);
    }
     
    public boolean handleFault(LogicalMessageContext context) 
    {
        return processMessage(context);
    }
     
    public void close(MessageContext context) 
    {
        // Clean up Resources
    }	

    private boolean processMessage(LogicalMessageContext context) 
    {
    	Boolean outboundProperty = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
    	
    	if (outboundProperty) 
    	{
    		// System.out.println("\nOutbound message:");
        	Map<String, List<String>> httpRequestHeaders = (Map<String, List<String>>) context.get(MessageContext.HTTP_REQUEST_HEADERS);

            if (httpRequestHeaders == null)
            {
            	httpRequestHeaders = new HashMap<String, List<String>>();

            	context.put(MessageContext.HTTP_REQUEST_HEADERS, httpRequestHeaders);
            }
            if (oauthBearerToken != null)
            {

            	httpRequestHeaders.put("Authorization", oauthBearerToken);
            }
    	} 
    	return true;
    }

    public void setOAuthBearerToken(String oauthToken)
    {
    	oauthBearerToken = new ArrayList<String>();
    	oauthBearerToken.add("Bearer " + oauthToken);
    }
}

    /**
     * Configure session attributes and handlers for given service
     * 
     * @param wsdlProvider - binding provider for service
     * @param strServiceURL - specific URL for service configured
    **/
    static void configureServiceSession(BindingProvider wsdlProvider, String strServiceURL)
    		throws MalformedURLException, IOException
    {
    	// configure requestContext - WSDL URL and Session attribute
    	Map<String, Object> requestContext = wsdlProvider.getRequestContext();

		requestContext.put( BindingProvider.ENDPOINT_ADDRESS_PROPERTY, 
		                                    "http://"" + _server + ":" + _port + strServiceURL );
        requestContext.put(BindingProvider.SESSION_MAINTAIN_PROPERTY, Boolean.TRUE);
		
        // Handle OAUTH2 Client
		TSSWSAPIHandler tssWSAPIHandler = new TSSWSAPIHandler();
        tssWSAPIHandler.setOAuthBearerToken(getOAuthToken("http://"" + _server + ":" + _port + strSpotfireServerBase));
	
		List<Handler> handlerChain = new ArrayList<Handler>();
        handlerChain.add(tssWSAPIHandler);
		Binding bindObj = wsdlProvider.getBinding();
        bindObj.setHandlerChain(handlerChain);
    }

    
    /**
     * Calls TSS OAuth Token URL to get Bearer authentication access token
     * Uses previously configured oAuthClientID and oAuthClientSecret
     * 
     * @param strServerURL - base server URL
    **/
    static String getOAuthToken(String strServerURL)
    {
        // for this example only need the scope - "api.soap.update-analysis-service"
        /*
        C:\tibco\tss\7.13.0\tomcat\bin>config register-api-client -n TestUpdateAnalysis -Sapi.soap.update-analysis-service

        Tool password:
        Successfully registered a new API client with the display name 'TestUpdateAnalysis':
        Client ID: <YOUR CLIENT ID>
        Client secret: <YOUR CLIENT SECRET>
        To view the full client configuration, please use the 'show-oauth2-client' command.
        */

        // New with api.soap.impersonate
        String oAuthClientID = _clientID;
        String oAuthClientSecret = _clientSecret;
        String accessToken = null;

        String urlOAuth = strServerURL + "/oauth2/token";

        try
        {
            System.out.println("Retrieving OAuth Token from: " + urlOAuth);

            URL wrOAuth = new URL(urlOAuth);
            HttpURLConnection wrInitial = (HttpURLConnection)wrOAuth.openConnection();

            String oAuthClientInfo = URLEncoder.encode(oAuthClientID, "UTF-8") + ":" + URLEncoder.encode(oAuthClientSecret, "UTF-8");
            String base64OAuth = Base64.getEncoder().encodeToString(oAuthClientInfo.getBytes("UTF-8"));

            wrInitial.setRequestProperty ("Authorization", "Basic " + base64OAuth);
            wrInitial.setRequestMethod("POST");
            wrInitial.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=UTF-8");
            
            String strScopeInfo = "grant_type=client_credentials&scope=" + URLEncoder.encode("api.soap.update-analysis-service", "UTF-8");
            byte[] postBytes = strScopeInfo.getBytes("UTF-8"); 

            wrInitial.setRequestProperty("Content-Length", "" + postBytes.length);
            wrInitial.setUseCaches(false);
            wrInitial.setDoInput(true);
            wrInitial.setDoOutput(true);   
            wrInitial.getOutputStream().write(postBytes);

            // read response information from request
            InputStreamReader responseStream = new InputStreamReader(wrInitial.getInputStream(), "UTF-8");

            Scanner sc = new Scanner(responseStream).useDelimiter("\\A");
            String input = sc.hasNext() ? sc.next() : "";
            // parse out JSON return data
            ObjectMapper javaScriptSerializer = new ObjectMapper();

            Map<String, Object> dictionary = (Map<String, Object>)javaScriptSerializer.readValue(input, new TypeReference<Map<String,Object>>(){});
            for (Map.Entry<String, Object> current : dictionary.entrySet())
            {
                String key;
                if ((key = current.getKey()) != null)
                {
                    if (key.equalsIgnoreCase("access_token"))
                    {
                        accessToken = (String)current.getValue();
                        continue;
                    }
                    if (key.equalsIgnoreCase("token_type"))
                    {
                        continue;
                    }
                    if (key.equalsIgnoreCase("expires_in"))
                    {
                        continue;
                    }
                }
            }
        }
        catch (Exception ex)
        {
            System.err.println("Exception calling OAuth Token URL, " + urlOAuth + ": " + ex.getMessage());
        }
        return accessToken;
    }

One needs to call the configureServiceSession function to configure the authentication for the OAuth 2.0 protocol.  Now the call to the Web Service becomes the following:

/**
 * 
 */
package com.tibco.spotfire.webservices.UpdateAnalyisService;

import com.tibco.spotfire.ws._2015._08.externalscheduledupdate.UpdateAnalysisServiceException;
import com.tibco.spotfire.ws._2015._08.externalscheduledupdate.UpdateAnalysisServiceImpl;
import com.tibco.spotfire.ws._2015._08.externalscheduledupdate.UpdateAnalysisServiceImplService;

import java.net.Authenticator;
import java.net.MalformedURLException;
import java.net.PasswordAuthentication;
import java.net.URL;

import com.tibco.spotfire.ws._2015._08.externalscheduledupdate.LoadAnalysis;
import com.tibco.spotfire.ws._2015._08.externalscheduledupdate.LoadAnalysis.UpdateAnalysis;

/**
 * @author Andrew Berridge, TIBCO Data Science
 *
 */
public class Test {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		try {
			
			//
			// Create the Library Service instance
			//
			WebServiceClient annotation = UpdateAnalysisServiceImplService.class.getAnnotation ( WebServiceClient.class );  

			/***********************************************************************************/
			/* create new update analysis service and get authentication tokens               */
			/***********************************************************************************/
			UpdateAnalysisServiceImplService updateAnalysisService = new UpdateAnalysisServiceImplService( 
					new URL ( "http(s)://spotfireserver:port/spotfire/api/soap/UpdateAnalysisService/wsdl" ),  
					new QName ( annotation.targetNamespace (), annotation.name () ) );
			
			UpdateAnalysisServiceImpl test = updateAnalysisService.getUpdateAnalysisServiceImplPort();

			//
			// Authentication done via OAuth
			// code in this function handles the OAuth 2.0 protocol
			configureServiceSession((BindingProvider)test, strUpdateAnalysisServiceEndpoint);
			
			/* update analysis   */
			UpdateAnalysis ua = new UpdateAnalysis();
			ua.setPath("/Path to My Analysis");
			test.getUpdateAnalysisServiceImplPort().loadAnalysis(ua);
			
		} catch (MalformedURLException | UpdateAnalysisServiceException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}
}

tss_updateanalysisservice_csharp (1).zip

tss_updateanalysisservice_java (1).zip