AI - LLM Integration

Bring Your Own Token (BYOT) - Recommended

We support all the major LLM providers (see this list) as well as existing LLM deployments within your organization. We recommend Google Gemini models. Edit your values.yaml configuration with the following:

ai:
  enabled: true
  apiKey: "<llm-provider-api-key>" # Leave empty if no authentication is needed
  models:
    visionModel: # Needs to be vision capable!
      name: "gemini/gemini-3-pro-preview"
    miniModel:
      name: "gemini/gemini-3-flash-preview"
    nanoModel:
      name: "gemini/gemini-3-flash-preview"
  
  # Optional settings
  ## ai.apiBaseUrl: when unset, the default url for the provider will be used.
  ## change if you use a different provider (example: openai on azure, llama4 on own infrastructure) 
  apiBaseUrl: "my-base-url"

Embedded Deployment

You will need to set the values below and pass thegpuflag to install.sh

Values:

This example shows how to locally install the gemma3:27b model

ai:
  enabled: true
  localDeployment: true
  apiBaseUrl: "http://order-management-ollama:11434"
  apiKey: ""
  models:
    visionModel:
      name: "ollama/gemma3:27b"
    miniModel:
      name: "ollama/gemma3:27b"
    nanoModel:
      name: "ollama/gemma3:27b"

ollama:
  persistentVolume:
    enabled: true
    size: 150Gi
  extraEnv:
    - name: OLLAMA_KEEP_ALIVE
      value: "9999999h"
    - name: OLLAMA_CONTEXT_LENGTH
      value: "12288"

Pulling the models:

You have two options to pull the gemma3:27bmodel:

With internet connection:
1. Once installed, shell into the ollama container and issue ollama pull gemma3:27b
Without internet connection:
1. On a separate machine with connection, install Ollama and ollama pull gemma3:27b
2. This will create a ~/.ollamaor C:/Users/username/.ollama directory with the model. Transfer this folder to the server where the 3yourmind installation is.
3. Once the 3yourmind installation installation is succeeded, copy the content of the transferred .ollama folder to the ollama persistent volume.
4. You can find the location of the localStorage mount path by describing the ollama persistent volume (it should look like/var/openebs/local/pvc-d9df436d-d873-4e42-a0fc-5381157a908c). There should be an empty .ollamafolder which is the target for replacement.

Running and testing the model:

Shell into the ollama container if not already there
Load the model and optionally test it with ollama run gemma3:27b
Leave the ollama session with /bye and check that the model is still running in the background withollama ps