How to connect SQL Server data with TIBCO Spotfire using Live Datamart (Spotfire Data Streams) for live data processing

[Balakumar Krishnasamy]

Hi All,

I am having SQL server and Hive data to be used with TIBCO spotfire for data visualization. But, it has to be streaming one.

I am aware that it has to be connected via Liva Datamart (Spotfire Data Streams). I am not good with Liva Datamart.

Someone please explain how this connectivity has to be created

From Live Datamart,

How these connectivity can be extablished

Do we have adapters to connect directly Or we have to use custom adapter

Regards,

Balakumar Krishnsamy

[Nawaz Ali Mohammad]

Hi Balakumar,

 

What version of Spotfire are you using. Starting from Spotfire Analyst version 10.0, there is a connector in Spotfire Analyst (Data Sreams) using which you can connect to Live Data. Refer to screenshot.

[Steve Barber]

KrishNerd --

To pursue this topic, it would be helpful I think to clarify what you mean here bystreaming. You mention both SQL Server and Hive as data sources for your visualizations. Neither SQL Server nor Hive typicallycreate streams as output -- generally you issue a static query against one of these and get back a result set, and a result set is not a stream. So what about your visualization would be visualizing a stream A stream of what

Yes, Spotfire Data Streams can essentially proxy data from any datasource that presents a JDBC interface -- and both (Microsoft) SQL Server and Hive tables can be queried via JDBC. So one could set up a LiveView Data Table in a Spotfire Data Streams/Live View server, and have that table be populated from a SQL Server or Hive table from a LiveView Embedded Publisher or Application Data Source. That application could use a Metronome operator to periodically poll the EventFlow JDBC Query to update the LiveView data table with any changes in the table. That would work. Polling isn't really streaming, but you wouldn't be the first to want to do something like this.

For Microsoft SQL Server, there is aChange Data Capture Input Adapter that would work to listen for changes in the database and push them to your LiveView Embedded Publisher or Application Datasource. Which would then use those new records to update a LiveView Data Table. That's pretty streamy -- basically you are driving your visualization off a variant of the table's transaction log.

But maybe you have a different idea than either of those as to what you mean bystreaming.

Once you've got either or thosegoing, the answer here from Nawaz is correct: create an instance of the Spotfire Data Streams Connector in Spotfire and then create your Spotfire visualization from that.

So, it's not so much that you need to create a custom StreamBaseadapter, but you do need to know how to present your data as a stream that will eventually drive aSpotfire visualization, and how to fit the existing pieces of that together inside the LiveView server. So it would be fair to sayyou have to write a custom LiveView application to do what you want to do. In order to do that, it would help to learn more about LiveView and StreamBase's EventFlow language, and its componentsand adapters.

To that end, we have a lot of different learning materials available. We have an overview course here:Streaming Analytics using StreamBase10.xif you are interested in pursuing these topics.