How can I delete a data canvas operation with an iron python script?

[Ryan James 5]

I have a data function that adds rows to a table. I'd like to be to delete selected rows from that data table with an iron python script that deletes that actual data canvas operation rather than just the table row. If just the table row is removed and not the operation the row reappears on the next reload.

Thanks,

[Fredrik Rosell]

Hello Ryan,

Ignoring the IronPython part to start with, just to confirm that I understand the use case, it sounds like you should add a Filter Rows transformation. In my example below I have the following data

Col1 Col2

1 2

2 5

3 7

4 6

5 9

6 10

I now add a Filter Rows transformation to remove the rows where Col1 = 3 or 4:

[Col1] not in (3,4)

 

From then on, even when you reload the analysis and the underlying data has changed, it will filter out Col1 = 3 or 4

In this example, I end up with:

Col1 Col2

1 2

2 5

5 9

6 10

Does this sound like what you need?

[Ryan James 5]

Hi Fredrik, That method would be a temporary solution but it would be less preferred as it still leaves all the steps in the data canvas. This workbook will probably be used by many people and there could be hundreds of added lines. I could accomplish the same by manually removing the data canvas node.

I'd prefer to find a solution that allows me to use a property control and an iron python script (button in a text area) to permanently remove uneeded rows.

[Fredrik Rosell]

Ok, I wasn't sure about the part about "I'd like to be to delete selected rows from that data table" (if that was about marking *any* rows in the table). So is it rather about removing entire Add Rows operations? Sorry for all the questions - I think I still don't understand the exact use case.

[Ryan James 5]

Yes, that's it exactly, it's about removing entire add rows operations. The idea is that this table will be around for a long while, but occaisonally previous additions will need to be removed to clean up the data set before exporting or the ability to backtrack if mistakes are made. I was thinking one of two options:

1. Select rows to delete with a document property (drop down list or multi-select)
2. Mark rows to delete in table

The click a button to execute a script to delete the selected rows and 'add rows operations'.

Is that doable?

Thanks,

Ryan

[Gaia Paolini]

If you wish to remove operations, you need to work at the level of individual operations, in the sense that each operation will have deleted a number of rows, but it is not possible to look inside the operation and pick the rows you want to un-delete.

I am attaching a dxp with a script that lets you remove the last operation of this type, recursively, until there are no more operations to remove. I tried to put it in a loop, but it did not work, and it is sensitive to the order of operations (if you try to remove one in the middle, it complains).

So you have to keep pressing it until the number of operations of this type is zero (I added a document property for convenience).

The script is:

from Spotfire.Dxp.Data.DataOperations import DataSourceOperation
from Spotfire.Dxp.Data.DataOperations import DataOperation
 
#tbl is an input parameter of type Data Table.
 
#Remove last RemoweRows operation
sourceView = tbl.GenerateSourceView();
op=sourceView.LastOperation
if type(op).__name__ == "RemoveRowsOperation" and \
	sourceView.CanRemoveOperation(op):
		sourceView.RemoveOperation(op)
 
 
#Find how many of these operations are left
sourceView = tbl.GenerateSourceView();
allDataOps = sourceView.GetAllOperations[DataOperation]()
numRemoveRowsOperations=0
for op in allDataOps:
	if type(op).__name__ == "RemoveRowsOperation":
		numRemoveRowsOperations=numRemoveRowsOperations+1
 
#The document properties needs to exist already (type integer)
Document.Properties['numRemoveRowsOperations']=numRemoveRowsOperations

 

delete_removerows_transformation.dxp

[Ryan James 5]

Thanks very much, I'll give it a shot!

Sounds like it's going to be a partial solution though, because of the sequence. That's ok, this is still useful, just sounds like the full solution just isn't possible with current functionality.

Thanks again.

[Gaia Paolini]

I found now what looks like a complete solution. I was reminded that removing rows is a destructive operation, in the sense that a new sourceview is generated and the old one becomes invalid.

 

from Spotfire.Dxp.Data.DataOperations import DataSourceOperation
from Spotfire.Dxp.Data.DataOperations import DataOperation
 
#tbl is an input parameter of type Data Table.
 
## functions
def find_remaining_operations(sv):
	allOps = sv.GetAllOperations[DataOperation]()
	numOps=0
	for op in allOps:
		if type(op).__name__ == "RemoveRowsOperation":
			numOps=numOps+1
	return numOps
 
###---------------------------------------------
#Remove last RemoweRows operation
sourceView = tbl.GenerateSourceView()
 
#remove operation is destructive and sourceView is updated after every remove
while find_remaining_operations(sourceView)>0:
	op=sourceView.LastOperation
	if type(op).__name__ == "RemoveRowsOperation" and \
		sourceView.CanRemoveOperation(op):
			sourceView = sourceView.RemoveOperation(op)
 
 
#Find how many of these operations are left - this is now just optional
#The document properties needs to exist already (type integer)
numRemoveRowsOperations=find_remaining_operations(sourceView)
Document.Properties['numRemoveRowsOperations']=numRemoveRowsOperations