Spotfire report with scheduled updates

[Manish Kumar 11]

Hi all,

We have a database table (single table with around 8 columns)that stores the transactions for a business day. Each day, we process around 500K transactions and we have data for last 3 months totalling to 50M+ rows. Using this as data source, we created a statistics report that shows few reports based on transaction type and transaction amount. We also have some reports wherein we show the daily no of transactions etc., Every day, the underlying data size is increasing and while loading the report, we want to see the latest information (till yesterday). So we set a scheduler to run every night to go and update the information link (the time is post the load of daily data to the database). When we check the report next day, we intend to see results of previous day as well, but it doesn't seem to be working. We check the logs in the scheduler and find that the rule was executed successfully, however as an end user, we do not see the expected results. FYI - We have set this report's data loadas "New data always". We want to know the best combination to set to get the desired results.

Data load - Should it be "New Data always", "Cached data" or "New data when available"

Scheduler - How does it correlate to the data load settings

Cached data - Is the SBDF file stored on the physical file system. Should we go for this option knowing that we have a ever growing data size.

SF processes - The SF services seem to consume upto 10GB of RAM during the load of this report, anything to do with the data load or scheduler

Regards

MKV

[Aditya Singh 3]

Since you have set "New Data always", so your analysis is not retrieving cached data and trying to load new data whenever the report is opened in webplayer. You need to unckeck new data always option and refresh the cache schedule to see th cachced data.

[Manish Kumar 11]

Dear Addy,

 

Thanks for your answer. Based on your response, we set the analysis to Cached option and saved the file. The first time we opened the analysis in web player, it took time to load 50M+ of data. However, post that it was quick.

 

Next day, on the data side, we had an increment of around 500K of data which means the datasize was now 50.5M. Post loading data into database, we set a scheduler to update the report. We checked the activity logs and found that the rule was triggered and the status was "Load". It completed it in around less than 6 mins.

 

Our expectation was, when we reloaded the analysis file in the web player (.html), we intended to see the most updated data (assuming scheduler would have refreshed the cache with new data), however this is not the case. We still see the 50M+ rows and now the additional data that we loaded in the data source (500K) is not coming up. Unable to understand the link between cache setting and use of schedulers/rules.

 

Regards

 

MKV

[Trent Foley]

I am experiencing the same issue as MKV.  Scheduler shows the file to have reloaded successfully, but upon opening the analysis in the Web Player, I do not see updated data.  What are we missing