Spotfire data size limits (1.02 billion row table)

[Rich Gordon 2]

Having some issues loading one of my datafrom a SQL View. It is not loading all 1.02 billionrows.

1. Any recommendations What is Spotfire's maximumloadable rows

2. Which would be better load proceedure: Import Data Table or Keep data table external

On some larger files, I noticed difficulty getting my filter scheme using keep table external.

I'd like to avoid creatingsmaller data sets to analyze different analysis components because of fewer drill down opportunties

 

Version: Tibco Spotfire Analyst 7.8.0.1.20

[Sean Riley]

Spotfire does not have a hard coded limit on the size of data or number of rows, instead you are limited on your system resources. If you are working with 1.02 billion rows then you will almost certainly have to 'Keep data table external' so that you only pull in aggregated results. Otherwise, if you try and import the data table, you will load all 1.02 billion rows of data in your local client RAM, which I assume will go very badly since your RAM will be consumed, paging to disk will occur, and your system will begin to crawl.

 

Update:

To keep data table external.:

That is defined when you first create your data table, in the 'Load Method' option. Here is a reference to that dialog:

 

https://docs.tibco.com/pub/sfire-analyst/7.8.0/doc/html/en-US/TIB_sfire-...

 

 

That is where you set Load Method = Keep data table external. Note, this can only be done when first adding the table, it is not possible to change after the table has been added by editing the data table properties.

[Jawad Husseini]

Hi Sean,

 

What is the requierments to keeo the data table external and do you have any instructions to setup this feature

 

Cheers

 

Jawad

[Sean Riley]

That is defined when you first create your data table, in the 'Load Method' option. Here is a reference to that dialog:

 

https://docs.tibco.com/pub/sfire-analyst/7.8.0/doc/html/en-US/TIB_sfire-analyst_UsersGuide/add/add_details_on_add_data_tables.htm

 

[[{"fid":"112766","view_mode":"default","fields":{"format":"default","field_file_image_alt_text[und][0][value]":"","field_file_image_title_text[und][0][value]":""},"type":"media","attributes":{"height":"512","width":"562","class":"media-element file-default"}}]]

 

That is where you set Load Method = Keep data table external. Note, this can only be done when first adding the table, it is not possible to change after the table has been added by editing the data table properties.

[Abubakar anwar]

Hi all,

 

in reference to the problem above. I created a dataset using SAS. Later on I pushed the data into Spotifre library and created an information link for that.

the dataset contains 32 million rows.

Now i have to add this dataset to an existing dataset (which is already loaded in Spotfire environment) 93,000 rows, using a left join.

 

Is there a way to do it. I checked and when i use data from information link, it doesn't allow me to use the option of "Keep Data Table External". It is greyed out. Can anyone help me with it. It is very important for me to fix it otherwise a lot of important work will be discarded.

 

br,

Abu