AI - LLM Integration

Bring Your Own Token (BYOT) - Recommended

We support all the major LLM providers (see this list) as well as existing LLM deployments within your organization. We recommend Google Gemini models. Edit your values.yaml configuration with the following:

ai:
  enabled: true
  apiKey: "<llm-provider-api-key>" # Leave empty if no authentication is needed
  models:
    visionModel: # Needs to be vision capable!
      name: "gemini/gemini-3-pro-preview"
    miniModel:
      name: "gemini/gemini-3-flash-preview"
    nanoModel:
      name: "gemini/gemini-3-flash-preview"
  
  # Optional settings
  ## ai.apiBaseUrl: when unset, the default url for the provider will be used.
  ## change if you use a different provider (example: openai on azure, llama4 on own infrastructure) 
  apiBaseUrl: "my-base-url"

Embedded Deployment

We recommend qwen3-vl:32b-instruct or gemma3:27b as a good tradeoff between hardware requirements and performance.

You will need to set the values below and pass thegpuflag to install.sh

Values:

This example shows how to locally install the qwen3-vl:32b-instruct model

ai:
  enabled: true
  localDeployment: true
  apiBaseUrl: "http://order-management-ollama:11434"
  apiKey: ""
  models:
    visionModel:
      name: "ollama/qwen3-vl:32b-instruct"
    miniModel:
      name: "ollama/qwen3-vl:32b-instruct"
    nanoModel:
      name: "ollama/qwen3-vl:32b-instruct"

This example shows how to locally install the gemma3:27b model, which does not support dynamic resolution when deployed via ollama, and thus requires tiling of the input images

ai:
  enabled: true
  localDeployment: true
  apiBaseUrl: "http://order-management-ollama:11434"
  apiKey: ""
  tiling:
    enabled: true
    size: 896 
  models:
    visionModel:
      name: "ollama/gemma3:27b"
    miniModel:
      name: "ollama/gemma3:27b"
    nanoModel:
      name: "ollama/gemma3:27b"

Pulling the models:

You have two options to pull the ollama models. Say you want the gemma3:27b model:

With internet connection:
1. Once installed, shell into the ollama container and issue ollama pull gemma3:27b
Without internet connection:
1. On a separate machine with connection, install Ollama and ollama pull gemma3:27b
2. This will create a ~/.ollamaor C:/Users/username/.ollama directory with the model. Transfer this folder to the server where the 3yourmind installation is.
3. Once the 3yourmind installation installation is succeeded, copy the content of the transferred .ollama folder to the ollama persistent volume.
4. You can find the location of the localStorage mount path by describing the ollama persistent volume (it should look like/var/openebs/local/pvc-d9df436d-d873-4e42-a0fc-5381157a908c). There should be an empty .ollamafolder which is the target for replacement.

Running and testing the model:

Shell into the ollama container if not already there
Load the model and optionally test it with ollama run gemma3:27b
Leave the ollama session with /bye and check that the model is still running in the background withollama ps