Using On-Premises LLM with Spotfire Copilot: Configuration and Deployment

[Naoki Nakamichi]

Hello,

I am looking for guidance on how to configure and deploy an on-premises LLM (e.g., Llama3) with Spotfire Copilot. Specifically, I have two main questions:

1. How should the .env file be configured if we want to use an on-premises LLM instead of a cloud-based LLM like OpenAI?
2. What are the necessary steps and requirements to deploy the on-premises LLM as a service to integrate it with Copilot?

Any detailed instructions, best practices, or examples would be greatly appreciated.

Thank you!

[Prashant Maske]

Hello Naoki,

First of all you, will need to run the model locally and then make it accessible as a web service that Spotfire Copilot can communicated with. Ensure that your Spotfire Copilot instance is configured to point to the LLM service URL specified in the .env file.

Thanks