Appending data to existing table, results in execution time grow exponentially.

[Sumeeth Tewar]

Hello,

I have used the following Ironpython code snippet to append external csv to the spotfire data table.

 

defappendDataToTable(csvFileLoc, table):

if(os.path.exists(csvFileLoc)):

readerSettings=TextDataReaderSettings();

readerSettings.Separator=";";

readerSettings.AddColumnNameRow(0);

dSource=TextFileDataSource(csvFileLoc,readerSettings);

ifnotDocument.Data.Tables.Contains(table):

#iftabledoesntexist,createtableandimportdata.

tableNew =Document.Data.Tables.Add(table,dSource);

tableSettings=DataTableSaveSettings(tableNew,False,False);

Document.Data.SaveSettings.DataTableSettings.Add(tableSettings);

else:

#iftableexists,readdataandappendtodatatable.-

tableNew =Document.Data.Tables[table];

tableSettings=AddRowsSettings(tableNew,dSource);

tableNew.AddRows(dSource,tableSettings);

 

Please note the use case demands that the data grows in this table. But as i notice over a period of executions the execution goes slower and slower

Can you please let me know is this the correct way to append data to a table As in the correct approach where table data can grow in GB's over a period of weeks.

Or this is a better approach to do this