Need help for Datasource Reload Programmatically.

[Vijesh]

Hi team,

I am working on a spotifre dashboard, trying to dynamically change the file path of the data source  i.e csv  files for each users and then reload the data for that table. Below is my script. But I encounter an error at line  op.Reload().

if op.CanReload():
    op.Reload()
    print("DataSourceOperations Reloaded Successfully !")

dynamicPath.py

from Spotfire.Dxp.Data.DataOperations import DataSourceOperation, DataOperation
from Spotfire.Dxp.Application import DocumentMetadata
from Spotfire.Dxp.Data import *
from Spotfire.Dxp.Data.Import import *
from System.IO import *
from Spotfire.Dxp.Data import *
from Spotfire.Dxp.Framework.Library import LibraryManager, LibraryItemRetrievalOption

# Define the path to the CSV files (adjust according to your file structure)
final_path=Document.Properties["filepath"]+str(Document.Properties["udrrunid"])+"_"+str(Document.Properties["masterid"])+"\\"
Document.Properties["finalpath"] = final_path
print("The value of the document property is:", Document.Properties["finalpath"])

# Construction of Paths to each CSV file
case_file = Path.Combine(Document.Properties["finalpath"], "case.csv")
demo_file = Path.Combine(Document.Properties["finalpath"], "demo.csv")
drug_file = Path.Combine(Document.Properties["finalpath"], "drug.csv")
reac_file = Path.Combine(Document.Properties["finalpath"], "reac.csv")
drug_seq_file=Path.Combine(Document.Properties["finalpath"], "drug_seq.csv")
therapy_seq_file=Path.Combine(Document.Properties["finalpath"], "therapy_seq.csv")
event_seq_file=Path.Combine(Document.Properties["finalpath"], "event_seq.csv")
smq_file=Path.Combine(Document.Properties["finalpath"], "smq.csv")
product_groups_file=Path.Combine(Document.Properties["finalpath"], "product_groups.csv")
event_groups_file=Path.Combine(Document.Properties["finalpath"], "event_groups.csv")

#print case_file 
print demo_file
#print drug_file 
#print reac_file 
#print drug_seq_file
#print therapy_seq_file
#print event_seq_file
#print smq_file
#print product_groups_file
#print event_groups_file

# Function to print the table name, type, and source
def print_info(table_name, source_type, source):
    print("Data Table:", table_name, "Type:", source_type, "Source:", source)

# Iterate through all data tables in the document
for tbl in Document.Data.Tables:
    sourceView = tbl.GenerateSourceView()
    ops = sourceView.GetAllOperations[DataOperation]()
    
    for op in ops:
        if isinstance(op, DataSourceOperation):
            t = op.GetDataFlow().DataSource
            if type(t).__name__ in ["TextFileDataSource"] and "case.csv" in t.FilePath:
                t.FilePath = case_file
                #print_info(tbl.Name, type(t).__name__, t.FilePath)
            elif type(t).__name__ in ["TextFileDataSource"] and "demo.csv" in t.FilePath:
                 t.FilePath = demo_file
            elif type(t).__name__ in ["TextFileDataSource"] and "drug.csv" in t.FilePath:
                 t.FilePath = drug_file
            elif type(t).__name__ in ["TextFileDataSource"] and "reac.csv" in t.FilePath:
                 t.FilePath = reac_file
            elif type(t).__name__ in ["TextFileDataSource"] and "drug_seq.csv" in t.FilePath:
                 t.FilePath = drug_seq_file
            elif type(t).__name__ in ["TextFileDataSource"] and "therapy_seq.csv" in t.FilePath:
                 t.FilePath = therapy_seq_file
            elif type(t).__name__ in ["TextFileDataSource"] and "event_seq.csv" in t.FilePath:
                 t.FilePath = event_seq_file  
            elif type(t).__name__ in ["TextFileDataSource"] and "smq.csv" in t.FilePath:
                 t.FilePath = smq_file
            elif type(t).__name__ in ["TextFileDataSource"] and "product_groups.csv" in t.FilePath:
                 t.FilePath = product_groups_file
            elif type(t).__name__ in ["TextFileDataSource"] and "event_groups.csv" in t.FilePath:
                 t.FilePath = event_groups_file
            
            if op.CanReload():
                op.Reload()
                print("DataSourceOperations Reloaded Successfully !")

# Reload all data tables to reflect the new data sources
for tbl in Document.Data.Tables:
    tbl.ReloadAllData()
    print("Reloaded All Data successfull")
    tbl.ReloadLinkedData()
    print("ReloadedLinked Datas successfully")
    Document.Data.Tables.RefreshOnDemandData()
    print("RefreshOnDemand Datas successfully")
    tbl.Refresh()

The idea here is to reload the DatasourceOperations so that it will load data for each datasource when the user access the dashboard.

Error:

Traceback (most recent call last):
  File "<string>", line 72, in <module>
StandardError: Exception has been thrown by the target of an invocation.

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Spotfire.Dxp.Data.Exceptions.ImportException: Failed to execute data source query for data source "drug". ---> System.IO.IOException: The device is not ready.

   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share)
   at Spotfire.Dxp.Data.Import.FileDataSource.OpenFileStream(String file, Boolean deleteFileOnClose)
   at Spotfire.Dxp.Data.Import.TextFileDataSource.CreateDataRowReader(IServiceProvider serviceProvider)
   at Spotfire.Dxp.Data.DataSourceConnection.ExecuteQuery2()
   --- End of inner exception stack trace ---
   at Spotfire.Dxp.Data.DataSourceConnection.ExecuteQuery2()
   at Spotfire.Dxp.Data.DataFlow.Execute()
   at Spotfire.Dxp.Data.DataFlow.DataFlowConnection.ExecuteQueryCore2()
   at Spotfire.Dxp.Data.DataSourceConnection.ExecuteQuery2()
   at Spotfire.Dxp.Data.Producers.SourceColumnProducer.<>c__DisplayClass76_0.<CreateView>b__0()
   at Spotfire.Dxp.Framework.ApplicationModel.Progress.ExecuteSubtask(String title, ProgressOperation operation)
   at Spotfire.Dxp.Data.Producers.SourceColumnProducer.CreateView(CxxSession session, DataPropertyRegistry propertyRegistry, GlobalMethodRegistry globalMethodRegistry, DataSourceConnection connection, IDataPropertyContainer defaultColumnProperties, PartialDataLoadReport& partialLoadReport)
   at Spotfire.Dxp.Data.Producers.SourceColumnProducer.GetColumnsAndProperties(DataSourceConnection connection)
   at Spotfire.Dxp.Data.Persistence.DataItem.PerformUpdate(SourceColumnProducer producer, DataSourceConnection connection)
   at Spotfire.Dxp.Data.Persistence.DataItem.Update(SourceColumnProducer producer, DataSourceConnection connection)
   at Spotfire.Dxp.Data.Persistence.DataPool.<LoadData>d__15.MoveNext()
   at Spotfire.Dxp.Internal.EnumerableExtensions.<EnumerateWithExceptionHandling>d__7`2.MoveNext()
   --- End of inner exception stack trace ---
   at Spotfire.Dxp.Data.DataTable.<>c.<Refresh>b__257_0(Exception e)
   at Spotfire.Dxp.Data.DataTable.BeginRefresh(Boolean reportColumnsChangedResultToNotificationService, Boolean async, Action`1 continuation, DisposableCollection disposableCollection, Predicate`1 isLeafAffected, Boolean refreshEmbeddedSources, DataLoadSettings loadSettings)
   at Spotfire.Dxp.Data.DataTable.Refresh(Boolean reportColumnsChangedResultToNotificationService, Predicate`1 isLeafAffected, Boolean refreshEmbeddedSources, DataLoadSettings loadSettings)
   at Spotfire.Dxp.Data.DataOperations.DataOperation.ReloadCore(DataLoadSettings loadSettings)
   at Microsoft.Scripting.Interpreter.ActionCallInstruction`1.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.LightLambda.Run3[T0,T1,T2,TRet](T0 arg0, T1 arg1, T2 arg2)
   at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)
   at Microsoft.Scripting.Interpreter.DynamicInstruction`3.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.Interpreter.Run(InterpretedFrame frame)
   at Microsoft.Scripting.Interpreter.LightLambda.Run2[T0,T1,TRet](T0 arg0, T1 arg1)
   at IronPython.Compiler.PythonScriptCode.RunWorker(CodeContext ctx)
   at Microsoft.Scripting.Hosting.ScriptSource.Execute(ScriptScope scope)
   at Spotfire.Dxp.Application.IronPython27.IronPythonScriptEngine.ExecuteForDebugging(String scriptCode, Dictionary`2 scope, Stream outputStream)

Kindly help to get this error fixed. 

Thanks in advance for suggestion!

 

Regards

Vijesh

Edited November 5 by Vijesh

[Jose Leviaguirre]

Hello Vijesh,

The error caused by op.Reload() indicates that the file path specified for one of the CSV files is not accessible or does not exist. You should add error handling to check if the file exists before attempting to reload the data source.