How can i create live view using my own data set in streambase

[Geetika Saini]

Hello,

I am new to tibco data streaming,can any one help me with the steps used to create live streaming data using my own dataset

[Steve Barber]

Hi,

I'm not sure exactly what you are looking to do, because your question is a very broad one.

But since all the other questions you've asked here on Community were about Spotfire, I will give you an example of how to stream a dataset kept in a CSV file into a Spotfire X table visualization, using TIBCO Spotfire Data Streams (or TIBCO Streaming LiveView server).

Here's a step by step guide to doing this:How to stream data from a CSV file with Spotfire Data Streams into a Spotfire X visualization

Of course, simulating a live stream of events using a CSV file may not be what you want to do. You may getting your data from an actual streaming source. You can go through a similar set of steps using the Connectivity Wizard as outlined on the CSV page above, if you are using TIBCO Enterprise Message Service (EMS), Apache Kafka, or MQTT, especially if the messages have a JSON payload.

If your data is from some other streaming data source, there will likely be some EventFlow coding involved: Ioutline some of those steps in my answer in this Answer post:Automatic refresh.

But before going further down that road in this answer post, I think it would be better if you had a look at what I've written here and pointed to, and then maybe if you want more information especially about integrating with different kinds of streaming data source or you were looking to do something else with streaming data apart from visualizing it in Spotfire X, perhaps you could respond here with more specifics, preferably in a comment to your original question.

[Geetika Saini]

Hello,

 

Thanks a lot for your response. Actually I am evaluating options for PostgresSQL database. My data resides in Postgres IF you have any idea how I can integrate it to stream base, Please let me know.

 

Appreciate your response

 

Regards,

 

Geetika

[Steve Barber]

So you want to use a relational table stored in a Postgres database as a source to simulate streaming data into a StreamBase EventFlow application Is that correct I feel like I am guessing a lot here; please feel free to elaborate what you are doing with some more specifics, because I can think of several ways that a StreamBase application can interact with a relational database.

 

So, we have choices.

 

There's a StreamBase thing called a feedsim (Feed Simulation), which we used in the CSV example I gave you already. In addition to reading from delimited files, the feedsim engine can also read data from anything that presents a JDBC API, including Postgres. More information about StreamBase Feed Simulations is available starting at the Using Feed Simulations page of the TIBCO Streaming documentation set,  and we have specific doc page called Feed Simulation with a JDBC Data Source, too. The feed simulation engine can be run in three different ways, too: 1) using the Feed Simulation Input Adapter, as we saw in the CSV example, 2) from the command line with the epdamin playback target, and 3) from StreamBase Studio's Feed Simulations view. Like I said, we have options, and which you choose depends on what it is you want to do. My experience with JDBC playback via feedsim, though, is that the feedsim engine may not be flexible enough to transform the data the way you might want it, so if you choose this method, you'll probably end up doing some data transformations in the EventFlow application itself.

 

The more integrated, and the more usual, way that StreamBase applications interact with relational databases is to use the EventFlow Query operator along with the JDBC Table construct. Using the Query operator, one can read, write, and delete relational table data, and translate the data into StreamBase tuples for further processing. An overview of the JDBC Query functionality is here.

 

The third way that people sometimes want to integrate relational database data with StreamBase applications is using some kind of Change Data Capture mechanism. Change Data Capture is a set of design patterns where updates to a database create a stream of events which one can think of, perhaps, as being roughly equivalent to listening to a transaction log. Change Data Capture isn't a standard of any kind, and there's no CDC API we could integrate with that would be good for all databases. We do have a CDC adapter that handles both Oracle Server and Microsoft SQL Server, but alas, nothing for the CDC paradigm for Postgres.

 

So, I think that covers the options. It's not my intention to just point you at the documentation set and send you on your way -- my intention at this point is to describe the options to help you be able to more fully articulate what it is you want to do in a more StreamBase-y way, and once I understand that, I poimt you to more specific examples, if that would be helpful.

 

 

[Geetika Saini]

Here are the details of my use case.

 

Spotfire X should be able to visualize data in real-time, meaning as and when new data is fed in third party Application, it should get reflected in the Spotfire Dashboard in almost real-time.

 

Solution-

 

My approach- Third-Party Application is been integrated using Tibco-BW and using APIs Data is been fed to PostgresDB and DB is then connected to Spotfire using Postgres Connector.

 

I am bringing the data in the memory of Spotfire by importing the data as I want to use custom visualization and utilize Spotfire functionality.

 

The problem I am facing

 

Refresh time for new Dataset on Spotfire is very high, it is taking more than 30 minutes to reload the Dashboards with new datasets and even getting crashed because of high Data volume.

 

The solution I am thinking-  Hence To overcome this load time, I am considering Tibco data Streaming so that we can have real-time Visualization on Spotfire Dashboard.

 

Looking for advice-

 

I hope I explained my use case in detail now.

 

I would like to know your opinion on the same, do you think Tibco data streaming is the solution I should explore for this thing to work and if so then What are the steps that I should take to make this work.

 

I am new to Data Streaming and have limited knowledge about Tibco Stream base product.

 

Thanks&Regards,

 

Geetika