Create a Custom Data Source in Spotfire®

Introduction

A custom data source is used to import data from any external data source into Spotfire?s in-memory data engine. When deployed in the Spotfire environment, the data source can be used to perform the following operations:

Add data table
Replace data table
Insert column to existing data table
Insert rows to existing data table

This tutorial provides the basics on how data sources load data into Spotfire as well as a simple blueprint for writing more advanced data sources.

Compared to Data Functions

A data source returns a single table without any input from the document and should be preferred for that simpler usecase. Data functions, in comparison, allow you to compute over multiple inputs from the document and output data to multiple places in the document. Because data sources return data row readers, and an ImportContext is available to the executor in the DataFunctionInvocation object, it is possible to wrap a data source inside a data function executor. See Wrap a Data Source inside a Data Function for more details.

Compared to Connectors

A connector can return multiple tables and use queries as input. It can also run this query in the external system. However, it still works without using data from the document as input.

Prerequisites

Spotfire® Developer (SDK), see download instructions here.
Spotfire® Analyst, download from edelivery.tibco.com.
Microsoft Visual Studio® 2013 or higher. The free Community edition is available for download here.

Overview

The extension point for the data source framework is the CustomDataSource class, which can be inherited from to implement a data source. This class is then registered in the RegisterDataSources method on the AddIn class together with a CustomTypeIdentifier. If the data source is concerned with a single file, derive from the CustomFileDataSource class instead.

Other classes in the framwork are:

DataSourceConnection - Represents an open connection to a data source. Either use a default instance or create a derived class.
CustomDataRowReader - Base class for custom data row readers. This can be used in, for example, a CustomDataSource or a CustomDataTransformation. It is also possible to create a default wrapper around an IDataReader by using one of the CreateReader methods on the base class, DataRowReader.

If the data source should have a UI that prompts for user input, a view class needs to be registered in the RegisterViews method on the Addin class. You can register view classes both for the Analyst client and the web client. For the Analyst client, a view class should inherit from System.Windows.Form. View classes for the web client should inherit from the PromptControl class.

Implementation

Create a CustomDataSource subclass:

namespace BasicDataSource
{
    using System;
    using System.Runtime.Serialization;
    using Spotfire.Dxp.Application.Extension;
    using Spotfire.Dxp.Data;
    using Spotfire.Dxp.Framework.Persistence;
 
    /// <summary>An example data source implementation.
    /// </summary>
    [Serializable]
    [PersistenceVersion(1, 0)]
    public sealed class BasicDataSource : CustomDataSource
    {
        /// <summary>Initializes a new instance of the BasicDataSource class.
        /// </summary>
        public BasicDataSource()
        {
            // Empty
        }
 
        /// <summary>Initializes a new instance of the BasicDataSource class.
        /// </summary>
        /// <param name="info">The serialization info.</param>
        /// <param name="context">The streaming context.</param>
        private BasicDataSource(SerializationInfo info, StreamingContext context)
            : base(info, context)
        {
            // Empty
        }
 
        /// <summary>Gets a value indicating whether the data source is linkable.
        /// </summary>
        public override bool IsLinkable
        {
            get { return true; }
        }
 
        /// <summary>Populates the SerializationInfo with the data needed to
        /// serialize the BasicDataSource instance.</summary>
        /// <param name="info">The serialization info.</param>
        /// <param name="context">The streaming context.</param>
        public override void GetObjectData(SerializationInfo info, StreamingContext context)
        {
            base.GetObjectData(info, context);
        }
 
        /// <summary>Creates a DataSourceConnection, using the specified context and
        /// the settings in this data source instance.
        /// </summary>
        /// <param name="serviceProvider">The service provider.</param>
        /// <param name="promptMode">The prompt mode.</param>
        /// <returns>The new DataSourceConnection instance.
        /// </returns>
        protected override DataSourceConnection ConnectCore(
            IServiceProvider serviceProvider,
            DataSourcePromptMode promptMode)
        {
            // Handle any prompting here. Not shown in this example.
 
            // Return the connection if all is OK.
            return DataSourceConnection.CreateConnection2(this, CreateReader, serviceProvider);
        }
 
        /// <summary>Creates the DataRowReader instance.
        /// </summary>
        /// <param name="serviceProvider">The service provider.</param>
        /// <returns>
        /// The DataRowReader instance.
        /// </returns>
        private static DataRowReader CreateReader(IServiceProvider serviceProvider)
        {
            return new BasicDataRowReader();
        }
    }
}

Next, subclass CustomDataRowReader, that will load the data rows into Spotfire. This example, BasicDataRowReader, returns two columns, one numeric and one text. A total of five rows of data will be loaded. The MoveNextCore method tells Spotfire whether there is more data available to read, and also handles the data population for the next row:

namespace BasicDataSource
{
    using System;
    using System.Collections.Generic;
    using Spotfire.Dxp.Application.Extension;
    using Spotfire.Dxp.Data;
 
    /// <summary>Implements CustomDataRowReader.
    /// </summary>
    internal class BasicDataRowReader : CustomDataRowReader
    {
        /// <summary>The number of rows of data available.
        /// </summary>
        private const int RowCount = 5;
 
        /// <summary>Holds the list of strings in the data.
        /// </summary>
        private readonly List<string> stringList;
 
        /// <summary>Holds the list of DataRowReaderColumn instances.
        /// </summary>
        private readonly List<DataRowReaderColumn> columns;
 
        /// <summary>Holds the index of the current row.
        /// </summary>
        private int currentRowIndex = 0;
 
        /// <summary>Initializes a new instance of the <see cref="BasicDataRowReader"/> class.
        /// </summary>
        public BasicDataRowReader()
        {
            this.columns = new List<DataRowReaderColumn>();
 
            // First column contains numbers.
            DataValueCursor column1Cursor = DataValueCursor.CreateMutableCursor(DataType.Integer);
            this.columns.Add(new DataRowReaderColumn("Numbers", DataType.Integer, column1Cursor));
 
            // Second column contains strings.
            DataValueCursor column2Cursor = DataValueCursor.CreateMutableCursor(DataType.String);
            this.columns.Add(new DataRowReaderColumn("Strings", DataType.String, column2Cursor));
 
            // Populate the data store for the second column.
            this.stringList = new List<string>() { "Zero", "One", "Two", "Three", "Four" };
        }
 
        /// <summary>The implementor should provide a list of <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s
        /// that it returns.
        /// </summary>
        /// <returns>The DataRowReaderColumn instances.</returns>
        /// <remarks>This method is only called once.</remarks>
        protected override IEnumerable<DataRowReaderColumn> GetColumnsCore()
        {
            return this.columns.AsReadOnly();
        }
 
        /// <summary>The implementor should provide the result properties.
        /// </summary>
        /// <returns>The ResultProperties instance.</returns>
        /// <remarks>This method is only called once.</remarks>
        protected override ResultProperties GetResultPropertiesCore()
        {
            return new ResultProperties();
        }
 
        /// <summary>Advance to the next row.
        /// The implementor should update all <see cref="T:Spotfire.Dxp.Data.DataValueCursor"/>s in
        /// the <see cref="T:Spotfire.Dxp.Data.DataRowReaderColumn"/>s with values for the next row.
        /// </summary>
        /// <returns>
        /// <c>true</c> if there are more rows; otherwise <c>false</c>.
        /// </returns>
        protected override bool MoveNextCore()
        {
            // Return false if trying to move past the last row.
            if (this.currentRowIndex == RowCount)
            {
                return false;
            }
 
            // Populate the data value cursors with new values.
            // First do the numbers.
            MutableValueCursor<int> intCursor = (MutableValueCursor<int>) this.columns[0].Cursor;
            intCursor.MutableDataValue.Value = this.currentRowIndex;
            intCursor.MutableDataValue.IsValid = true;
 
            // Now do the string.
            MutableValueCursor<string> stringCursor = (MutableValueCursor<string>) this.columns[1].Cursor;
            stringCursor.MutableDataValue.Value = this.stringList[this.currentRowIndex];
            stringCursor.MutableDataValue.IsValid = true;
 
            this.currentRowIndex++;
 
            return true;
        }
 
        /// <summary>The implementor should implement this method to reset the
        /// enumerator. If this method is called then the <see cref="M:Spotfire.Dxp.Data.DataRowReader.MoveNextCore"/>
        /// method should return the first row again when called the next time.
        /// </summary>
        protected override void ResetCore()
        {
            this.currentRowIndex = 0;
        }
    }
}

Before registering the new data source, define a new type identifier containing its name and description:

namespace BasicDataSource
{
    using Spotfire.Dxp.Application.Extension;
 
    /// <summary>Type identifiers used by the project.
    /// </summary>
    public class TypeIdentifiers : CustomTypeIdentifiers
    {
        /// <summary>The type identifier for our data source.
        /// </summary>
        public static readonly CustomTypeIdentifier BasicDataSource = CreateTypeIdentifier(
            "BasicDataSource",
            "Basic data source",
            "A very basic example data source which loads a few rows of data.");
    }
}

Finally, register the new data source using the AddIn class that must be part of every extension package:

namespace BasicDataSource
{
    using Spotfire.Dxp.Application.Extension;
 
    /// <summary>A very basic example data source which loads a few rows of data.
    /// </summary>
    public sealed class BasicDataSourceAddIn : AddIn
    {
        /// <summary>Register the data sources.
        /// </summary>
        /// <param name="registrar">The registrar.</param>
        protected override void RegisterDataSources(DataSourceRegistrar registrar)
        {
            base.RegisterDataSources(registrar);
 
            registrar.Register<BasicDataSource>(TypeIdentifiers.BasicDataSource);
        }
    }
}

Sign In

Create a Custom Data Source in Spotfire®

Introduction

Compared to Data Functions

Compared to Connectors

Prerequisites

See also

Overview

Implementation

Table of contents

User Feedback

Recommended Comments

Industries