How can I delete the Transformations that accumulate in the dataCanvas using ironPython script? (Transformations are accumulated after replacing columns in the table and the speed of the replacement will slow down.)

[Morio Saeki]

After replacing a column in a table using an ironPython script, Transformations will accumulate and the speed of the replacement will slow down. After I manually removed Transformations from the DataCanvas screen, the column update speed returned to normal.

'I used below ironPython script. 

1. from Spotfire.Dxp.Data import *
2. from Spotfire.Dxp.Data.Transformations import *
3.  
4. dataOperation = dataTable.GenerateSourceView().OperationsSupportingTransformations[0];
5. transformations = dataOperation.GetTransformations();
6.  
7. #clear all transformations on this dataTable if needed
8. dataOperation.ReplaceTransformations("")
9.  
10. #Replace specific value Transformation
11. column = DataColumnSignature("Name",DataType.String);
12. row_id_cols=[DataColumnSignature("No",DataType.Integer)]
13. row_id_col_values = [2]
14. transformations.Add(ReplaceSpecificValueTransformation(column, "chocolate1","chocolate2",row_id_cols, row_id_col_values,True))
15. dataOperation.ReplaceTransformations(transformations);

I referred to the sample below, and Spotfire Analyst uses Version 10.10.3 LTS

https://community.spotfire.com/s/article/Replace-specific-values-of-a-Data-Table-column-in-TIBCO-Spotfire-using-IronPython

I would like to remove the Transformations accumulated in the DataCanvas using an ironPython script so that the update speed does not decrease.(Or I would like to update columns in the table without slowing down)

​

thank you​

[Gaia Paolini]

if you remove the transformations, you also remove the specific substitutions and you are back to the original values. Are you sure you want to do that?

[Morio Saeki 3]

Yes.No problem.

The updated columns are temporary update, and there is no problem returning to the original value. Also, no other table updates are planned now.

[Gaia Paolini]

The Spotfire API groups transformations into operations. I could only find a way of deleting the entire operation, if it contains all transformations that replace cell values.

Can you try the script below?

The variable tbl is an input parameter (your data table).

from Spotfire.Dxp.Data.DataOperations import DataSourceOperationfrom Spotfire.Dxp.Data.DataOperations import DataOperationfrom Spotfire.Dxp.Data.Transformations import * #tbl is an input parameter of type Data Table.  ###--------------------------------------------- sourceView = tbl.GenerateSourceView()dataOpSupportingTransformations = sourceView.OperationsSupportingTransformations;  # Remove entire operation # if it only contains ReplaceValuesTransformation or ReplaceSpecificValueTransformation itemsreplaceValueTransformations=['ReplaceValuesTransformation','ReplaceSpecificValueTransformation']for op in dataOpSupportingTransformations: op0 = op.DataOperation op1 = op0.GetTransformations() are_they_replacements = [type(ttt).__name__ in replaceValueTransformations for ttt in op1] if all(are_they_replacements)==True and sourceView.CanRemoveOperation(op0): sourceView = sourceView.RemoveOperation(op0) 

[Morio Saeki 3]

I tried your script.

Result: not remove transformations.

Condition:3 Replace specific value exist.fruit1_condition.bmpI added below print() in your script.

print('dataOpSupportingTransformations',type(dataOpSupportingTransformations))

print('are_they_replacements',are_they_replacements)

[Output]

('dataOpSupportingTransformations', <type 'ReadOnlyCollection[DataOperationSupportingTransformations]'>)

('are_they_replacements', [True, True, True])

[estimation]

Not execute removeOperation in my environment ,because 'sourceView.CanRemoveOperation(op0)' return False.

Also I deleted 'sourceView.CanRemoveOperation(op0)' in if condition, and execute.

Result is [systemError: Cannot remove a source DataOperation if there are no other sources.]

Is it because 'dataOpSupportingTransformationsis' ReadOnly?

Please let me know if there is a way to prevent the settings from becoming ReadOnly. i will try.

thank you.

[Gaia Paolini]

can you share a sample dxp?

[Morio Saeki 3]

OK

fruits.dxp is my sample .dxp.

ColumnReplace: Column data replace Action button.

Sample: Action Button for your sample(I added 2 print line in your sample script)

SampleWithPrint: Script execute with many print line and some modify for my understanding.

[Gaia Paolini]

as far as I can initially see, your 'load data' is in the same operation as all the transforms. So you cannot remove that operation as it would remove all your data. In this case, sourceView.CanRemoveOperation(op0) is False, and the remove is not executed.

I am not sure of the solution, I am looking into it. In your real system, does this also happen?

You can verify by looking at the data canvas:

[Gaia Paolini]

I think I found a way:

from Spotfire.Dxp.Data.DataOperations import DataSourceOperationfrom Spotfire.Dxp.Data.DataOperations import DataOperationfrom Spotfire.Dxp.Data.Transformations import * #tbl is an input parameter of type Data Table. ###--------------------------------------------- sourceView = tbl.GenerateSourceView()dataOpSupportingTransformations = sourceView.OperationsSupportingTransformations;#print('dataOpSupportingTransformations',type(dataOpSupportingTransformations)) # Remove entire operation # if it only contains ReplaceValuesTransformation or ReplaceSpecificValueTransformation itemsreplaceValueTransformations=['ReplaceValuesTransformation','ReplaceSpecificValueTransformation'] for op in dataOpSupportingTransformations: op0 = op.DataOperation op1 = op0.GetTransformations() are_they_replacements = [type(ttt).__name__ in replaceValueTransformations for ttt in op1] try: if all(are_they_replacements)==True and op0.CanReplaceTransformations(): op0.ReplaceTransformations("") except: if all(are_they_replacements)==True and sourceView.CanRemoveOperation(op0): sourceView = sourceView.RemoveOperation(op0)

[Morio Saeki 3]

Data Canvas on my real sysytem is almos same as fruits.dxp.

The only difference is the number of ”replace specific value" and the csv file name.

I will try your new script and report the result.

[Morio Saeki 3]

​I successfully removed all the transformations in my real system using your script.

I will incorporate your Script's processing into my script.

​

Thank you for your cooperation.