Jump to content
We've recently updated our Privacy Statement, available here. ×

Spotfire report with scheduled updates

Manish Kumar 11

Recommended Posts

Hi all,

We have a database table (single table with around 8 columns)that stores the transactions for a business day. Each day, we process around 500K transactions and we have data for last 3 months totalling to 50M+ rows. Using this as data source, we created a statistics report that shows few reports based on transaction type and transaction amount. We also have some reports wherein we show the daily no of transactions etc., Every day, the underlying data size is increasing and while loading the report, we want to see the latest information (till yesterday). So we set a scheduler to run every night to go and update the information link (the time is post the load of daily data to the database). When we check the report next day, we intend to see results of previous day as well, but it doesn't seem to be working. We check the logs in the scheduler and find that the rule was executed successfully, however as an end user, we do not see the expected results. FYI - We have set this report's data loadas "New data always". We want to know the best combination to set to get the desired results.

Data load - Should it be "New Data always", "Cached data" or "New data when available"

Scheduler - How does it correlate to the data load settings

Cached data - Is the SBDF file stored on the physical file system. Should we go for this option knowing that we have a ever growing data size.

SF processes - The SF services seem to consume upto 10GB of RAM during the load of this report, anything to do with the data load or scheduler



Link to comment
Share on other sites

Dear Addy,


Thanks for your answer. Based on your response, we set the analysis to Cached option and saved the file. The first time we opened the analysis in web player, it took time to load 50M+ of data. However, post that it was quick.


Next day, on the data side, we had an increment of around 500K of data which means the datasize was now 50.5M. Post loading data into database, we set a scheduler to update the report. We checked the activity logs and found that the rule was triggered and the status was "Load". It completed it in around less than 6 mins.


Our expectation was, when we reloaded the analysis file in the web player (.html), we intended to see the most updated data (assuming scheduler would have refreshed the cache with new data), however this is not the case. We still see the 50M+ rows and now the additional data that we loaded in the data source (500K) is not coming up. Unable to understand the link between cache setting and use of schedulers/rules.





Link to comment
Share on other sites

  • 1 year later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...