Required Parameters for Orchestrator container

[Gerard Conway 2]

It is not at all clear what are the minimal set of required parameters that must be set to get a working Orchestrator container.

The parameter list has greatly increased since the previous copilot release.

Do the LANGCHAIN_... parameters need to be present and do they need to be set to legitimate values or can they be left blank?

Same question for the .._PLUGIN_.. parameters?

I have included them with dummy values and my container at least appears to start.

Also the Architecture Diagram in the Getting Started Guide does not show anything related to LANGCHAIN or PLUGINS.  I assume these are there to extend the functionality rather than run the default setup.

VECTOR_DB_TOKEN is also a new parameter.  I believe it's typically username:password combined for Milvus.  How is this handled if authentication hasn't been enabled on Milvus, will the VECTOR_DB_TOKEN value be ignored?

I'm using Milvus as the Vector Database and have successfully load the Spotfire docs. I've attached the attached the.env file I'm using for the Orchestrator which starts ok and is running.

I want to run a test query against the orchestrator REST endpoint.  However, I unsure of the parameter values required to execute a test query against the Vector Database.  Which parameters are required and which might be optional.

what should the index_name be? the collection name in Milvus i.e. spotfiredocs?

retriever type?

llm_name?  is this the model name to use e.g. gpt-3.5-turbo-1106?

llm_mode? I guess this is the new mode feature and should be 'Spotfire' for my test.

Assuming I have loaded only the enclosed Spotfiredocs what the most basic request body needed to run a test query.
 

Thanks,

Gerard

 

Orchestrator Container Error.txt

[Gerard Conway 2]

My Orchestrator is up and running.  I can run the root endpoint and it returns,
{"message": "DSML LLM Orchestrator - REST API"}

I've reviewed the OrchestratorRequest Schema in the Swagger API documentation but it's still not clear which parameters are required and which are optional.
When the inputs are (String | null) is null an acceptable value.  I'm doubtful of that for parameters like index_name.

Is retriever_type required?  What are valid values?  What is the appropriate value for a setup a milvus vector database with only spotfire docs loaded?

For llm_mode what is the best default, 'chat' or 'completions'?

Here is my current query (request body payload)

{
  "request_tag": "MyOrchRequest",
  "temperature": 0,
  "index_name": "spotfiredocs",
  "index_score_threshold": 0,
  "index_topk": 5,
  "llm_name": "gpt-3.5-turbo-1106",
  "llm_mode": "completions",
  "system_prompt": "Your are an AI assistant. Answer the question based on the context below. Keep the answer short and concise.",
  "user_intent": "QueryIntent",
  "history": null,
  "client_data": null,
  "prompt": "How to create a bar chart?"
}

It results in a error, KeyError: 'QueryIntent', see attached container log.

If I set user_intent to null, the error changes to KeyError: 'SpotfireHelp'.

Finally, if I try and use the Spotfire Copilot panel in Spotfire Analyst e.g. 'Ask Sample HowTo Question', I get the following single line error in the container log
POST / HTTP/1.1" 405 Method Not Allowed

Orchestrator Container Error QueryIntent.txt Orchestrator Container Error SpotfireHelp.txt

[Marcelo Gallardo]

@Gerard Conway 2

Thank you for providing the details about the issue you're experiencing. I understand that the increased number of parameters might be confusing. Let me address each of your concerns:

1. OrchestratorRequired Parameters:

   • The required parameters for configuring the Orchestrator container largely depend on the specific Large Language Model (LLM) and vector data store being utilized, which are. controlled by the plugin parameters
     • MODEL_PLUGIN_ENTRY_POINT
     • RETRIEVER_PLUGIN_ENTRY_POINT
   • The LANGCHAIN_... parameters are optional and are used to enable Langsmith. You can set the LANGCHAIN_TRACING_V2="false" and leave the other parameters blank if you don't want to use Langsmith
   • VECTOR_DB_TOKEN is indeed used for authentication purposes. If your Milvus setup does not require authentication, you can leave this parameter blank. The system should handle it gracefully. If authentication is enabled, this should be set to username:password.

2. Running a Test Query Against the Orchestrator REST Endpoint:

   • The endpoint is provided just to test very basic functionality. The documentation for each parameter can be seen by looking at the request schema in the swagger documentation.
   • To execute a simple test against the Orchestrator and make sure it is functional, you need to set the following parameters in your request body:

   • {
         "user_intent": "UserIntent",
         "prompt": "How to create a bar chart?"
     }

I hope this clarifies your questions. If you have any further doubts or need assistance with any specific configuration, feel free to reach out. We're here to help you get everything running smoothly.

 

Regards,

Marcelo Gallardo

 

[Gerard Conway 2]

Thanks @Marcelo Gallardo for the input.
Minimal query payload will be useful for testing.

However, using the attached .env file.  My Orchestrator container just cycles quickly between running and restarting and continues to do that.
I have attached the container log.

Am I doing something wrong in my .env.

Orchestrator Container Restarting.txt