Marcelo Gallardo

Niraj, Thank you for raising bringing up your client's concerns about data security when using large language models (LLMs). We take these concerns seriously and prioritize the protection of your data at every step. I would like to assure you that we follow strict security protocols when interacting with LLMs to mitigate any potential risks. Here are some key measures we have in place to ensure the security and privacy of your data: - Data Privacy: We take every precaution to ensure that sensitive data is only shared with LLMs when absolutely necessary. Any data processed is carefully managed, and we actively work to minimize the amount of data sent to the LLM. - Secure Transmission: All data interactions with LLMs occur over encrypted channels (using HTTPS), ensuring that no unauthorized parties can access the information during transmission. - No Data Retention: Many LLM providers, including OpenAI, have strict policies in place to ensure data is not stored after processing. We routinely review these policies with external vendors to ensure they meet these security standards. Furthermore, Copilot is LLM-agnostic, allowing you to select the LLM that best fits your data privacy requirements. Feel free to contact us if you need any additional information. We'd be happy to provide more details and assist you further. Regards, Marcelo Gallardo

@NirajThanks for the feedback and glad you resolved the issue.

@Niraj Could you please share the logs for the Orchestrator? Thanks, Marcelo

Niraj, We observed those issues in the past and were resolved by increasing the rate limit on the models used by Orchestrator. Also, do you have access to OpenAI? You could try that to see if you get the same rate limit issues. Thanks, Marcelo

Hello Niraj, Sorry you are having issues with copilot. Could you please provide the following information: - What version of the copilot orchestrator are you using? - Are you connecting to Azure OpenAI or OpenAI? - Can you access the Cognitive Search URL from the machine running orchestrator? Thanks, Marcelo Gallardo

@Gerard Conway 2 Thank you for providing the details about the issue you're experiencing. I understand that the increased number of parameters might be confusing. Let me address each of your concerns: OrchestratorRequired Parameters: The required parameters for configuring the Orchestrator container largely depend on the specific Large Language Model (LLM) and vector data store being utilized, which are. controlled by the plugin parameters MODEL_PLUGIN_ENTRY_POINT RETRIEVER_PLUGIN_ENTRY_POINT The LANGCHAIN_... parameters are optional and are used to enable Langsmith. You can set the LANGCHAIN_TRACING_V2="false" and leave the other parameters blank if you don't want to use Langsmith VECTOR_DB_TOKEN is indeed used for authentication purposes. If your Milvus setup does not require authentication, you can leave this parameter blank. The system should handle it gracefully. If authentication is enabled, this should be set to username:password. Running a Test Query Against the Orchestrator REST Endpoint: The endpoint is provided just to test very basic functionality. The documentation for each parameter can be seen by looking at the request schema in the swagger documentation. To execute a simple test against the Orchestrator and make sure it is functional, you need to set the following parameters in your request body: { "user_intent": "UserIntent", "prompt": "How to create a bar chart?" } I hope this clarifies your questions. If you have any further doubts or need assistance with any specific configuration, feel free to reach out. We're here to help you get everything running smoothly. Regards, Marcelo Gallardo

@Naoki Nakamichi Thank you for providing the details about the issue you're experiencing. Based on the information you shared, everything appears to be configured correctly. However, to further diagnose the issue, we need to review the Orchestrator container logs. We would appreciate it if you could share these logs with us. If you need any assistance with retrieving the logs, please let us know, and we’ll be happy to guide you through the process. Thank you for your cooperation and understanding. We look forward to resolving this issue promptly. Marcelo Gallardo

Alex, Orchestrator is trying to connect to your private Azure OpenAI endpoint. That is the error shown when the connection to the endpoint fails. I think the issue is that your private endpoint can't be accessed from the machine where Orchestrator is running. I will be happy to have a zoom and discuss the details. Thanks, Marcelo

Hello Alex, I will be happy to assist. Could you please let us know where is the container for the orchestrator running? It seems like it is deployed in a different machine from where your client browser is running. Please verify the network connection between the machine hosting the orchestrator and Azure OpenAI. You might have to manage the access to your Azure OpenAI endpoint and allow access to the machine hosting the orchestrator. Let us know if you need further assistance and we will happy to help. Regards, Marcelo Gallardo

@NirajGood to hear your issue got resolved. Please feel free to contact us with any other issue or suggestion. We will be delivering a new version that fixes a few issues in the next few days. Stay tuned. Marcelo

Hello @Niraj, Please make sure the password you are using matches the value used to obtain the hashed password as indicated in the instructions. Obtain a secret key which is used to sign the JWT tokens. You can obtain the key using openssl. (i.e. openssl rand -hex 32). Obtain a Bcrypt Hashed Password for the admin user. You can obtain a hashed password online from https://bcrypt-generator.com/. During the installation process, an admin user is set up with a bcrypt hashed password. This hashed password will be used to authorize access to the service’s administrative functions and for generating JWT (Json Web Token) tokens. The tokens will be used for authentication while accessing the service. Please save the password used as it will be needed to get the JWT token. Also note an issue related to Docker Compose when using environment variables that contain the “$” character, especially when working with hashed passwords. If your hashed password contains a “$”, please replace each “$” character with double “$$” to ensure that the password is correctly interpreted. Please let us know if this helps you resolve the issue. Marcelo Gallardo

@Gerard Conway 2 Thanks for your feedback and we are aware of the issue with OpenAI. Our next release coming in a few weeks addresses the issue. You can still try the current release if you have access to Azure OpenAI. Thanks for your patience. Marcelo.