Jump to content

python data function for large-scale data frame output speed issues


dochan

Recommended Posts

I'm a user of spotfire analysis version 10. Mainly when creating analysis solutions, spotfire data functions are used a lot, and sometimes large data frames have to be output.

Python data function seems to be a way to write data converted to sbdf format in a temporary directory and read it back from spotfire when outputting data to spotfire.

This is a significant time difference from when you output the same data with TERR. It took 1 minute for TERR and 22 minutes for Python to output 10 million rows of data frames.

Do you know how to improve these problems?
Or do I need to modify the source code of the 'spotfire' package?

Link to comment
Share on other sites

The Python performance is much improved since version 12.5. It is  hard to quantify the difference as it depends on the Python code, packages, the data, as well as the machine.
Some tests showed an average of 60% faster execution of Python data functions, often more.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...