Can I replace a transformation on-source via IronPython API?

[Mark Herrmann]

It is described how to replace a transformation applied to a data table after loading here.

The documentation also recommends to apply transformations after loading and not "on-source" unless you are working with "very large" data.

Since we are working with very large data I did some tests and the transformation "on-source" is consistently twice as fast. However, I need to be able to change the aggregation method of the applied pivot transformation via IronPython.

Is there any way this can be done for a transformation applied "on-source"? It can be edited in the data canvas so I hope the answer is "yes" but I can't find anything in the documentation...

Thanks,

Mark

[Andreas Laestadius]

It should work.

On a datasource, it is the "DataFlow" that is the important part, and when the transformations are replaced, it ought to reload immediately. (Same API calls, difference should be happening behind the scenes).

In the loop you'd have to extract the DataSourceOperation, not DataTransformationsOperation.

See also: https://docs.tibco.com/pub/doc_remote/sfire_dev/area/doc/api/TIB_sfire-analyst_api/html/M_Spotfire_Dxp_Data_DataOperations_DataSourceOperation_ReplaceTransformations.htm

[Mark Herrmann]

Great, thanks. Sounds like the hint I needed, will give it a try tomorrow.

[Mark Herrmann]

Not as straight forward as I had hoped...

For debugging I am using this loop for transformations applied after loading data from another table:

CRR_tmp=Document.Data.Tables['Correlation Table Frame-Lvl']sourceview=CRR_tmp.GenerateSourceView()i=0for op in sourceview.GetAllOperations[DataTransformationsOperation](): for t in op.GetTransformations(): i=i+1 print op.DisplayName,"#",i,":" print t.Name print op.CanReplaceTransformations()

This gives me a list with transformations and 'CanReplaceTransformations' is TRUE for all of them.

For transformations applied at load from another table I modified it as per your suggestion:

CRR_tmp=Document.Data.Tables['Correlation Table Die-Lvl']sourceview=CRR_tmp.GenerateSourceView()i=0for op in sourceview.GetAllOperations[DataTableDataSourceOperation](): for t in op.GetTransformations(): i=i+1 print op.DisplayName,"#",i,":" print t.Name print op.CanReplaceTransformations()

Now I also get a list with transformations but 'CanReplaceTransformations' is FALSE for all of them.

So I can read but not replace them? Is there another way to modify them but not via replacing them or am I misinterpreting the output?

Thanks, Mark

[Andreas Laestadius]

If CanReplaceTransformations() returns false, then calling DataTableDataSourceOperation.ReplaceTransformations will throw an InvalidOperationException. That exception will have a string message, indicating the reason why the DataTableDataSource node cannot be refreshed. The reason could give a clue as to what is the problem.

For example with a DataTable set to Embed the entire table, this code:

for op in sourceview.GetAllOperations[DataTableDataSourceOperation](): for t in op.GetTransformations(): i=i+1 print op.DisplayName,"#",i,":" print t.Name  print op.CanReplaceTransformations() newtrans = List[DataTransformation]() op.ReplaceTransformations(newtrans)

should throw:

T1 # 1 : Calculate new columnFalseTraceback (most recent call last): File "<string>", line 17, in <module>SystemError: Not possible since the owning DataTable is stored embedded in the analysis. System.InvalidOperationException: Not possible since the owning DataTable is stored embedded in the analysis. 

There are a few reasons that could prevent replace and there should be info the error message as to why in this case.

[Mark Herrmann]

Thanks, this will be really helfpul... in the future ;-) I had just deleted all my transformations and re-inserted them (switch off and back on strategy) when I saw your reply and guess what: after replacing the transformations with identical ones the problem is gone and CanReplaceTransformation() returns TRUE now for all as expected.

[Andreas Laestadius]

Great!

Thank you so much letting every body know.