vllm omni app

.github/workflows/build-push-images.yml

@@ -19,6 +19,7 @@ jobs:
           - chat
           - image-analysis
           - flux-image-gen
+          - omni
     permissions:
       contents: read
       id-token: write         # needed for signing the images with GitHub OIDC Token

.github/workflows/test-pr.yml

@@ -89,7 +89,7 @@ jobs:
         working-directory: web-apps
 
       - name: Install Helm
-        uses: azure/setup-helm@dda3372f752e03dde6b3237bc9431cdc2f7a02a2 # v5.0.0
+        uses: azure/setup-helm@1a275c3b69536ee54be43f2070a358922e12c8d4 # v4.3.1
 
       - name: Run base chart templating with default values
         run: helm template ci-test charts/azimuth-llm

.grype.yaml

@@ -0,0 +1,5 @@
+ignore:
+  # Patched in 7:7.1.4-0+deb13u1
+  - vulnerability: CVE-2026-40962
+    package:
+      location: /usr/bin/ffmpeg

README.md

@@ -22,6 +22,13 @@ This will install the latest stable [release](https://github.com/stackhpc/azimut
 
 Under the charts directory, there is a base [azimuth-llm](./charts/azimuth-llm) Helm chart which uses vLLM to deploy models from Huggingface. The [azimuth-chat](charts/azimuth-chat) and [azimuth-image-analysis](charts/azimuth-image-analysis) are wrapper charts which add different Gradio web interfaces for interacting with the deployed LLM.
 
+### Omni
+
+The [azimuth-omni](charts/azimuth-omni) chart deploys a single multimodal Gradio web interface (chat, text-to-speech and text-to-image) backed by one or more in-cluster [vLLM-Omni](https://github.com/vllm-project/vllm-omni) model deployments, provided by the [azimuth-omni-backend](charts/azimuth-omni-backend) subchart. Each modality (`ttt`/`tts`/`tti`) can be enabled independently and pointed at either an in-cluster or external backend.
+
+> [!IMPORTANT]
+> Unlike the base chart, the Omni backends are **GPU-only** — vLLM-Omni has no CPU image, so each enabled backend requires at least one NVIDIA GPU (`gpus >= 1`).
+
 ### Customisation
 
 The `charts/azimuth-llm/values.yaml` file documents the various customisation options which are available. In order to access the LLM from outside the Kubernetes cluster, the API and/or UI service types may be changed to

charts/azimuth-llm/templates/_helpers.tpl

@@ -77,6 +77,17 @@ Create the name of the service account to use
 {{- end }}
 {{- end }}
 
+{{/*
+Detect whether the configured API image is a vLLM Omni image.
+*/}}
+{{- define "azimuth-llm.isOmniImage" -}}
+{{- $image := .Values.api.image.containerImage | default "" -}}
+{{- $name := $image | splitList ":" | first -}}
+{{- if regexMatch "(^|/)vllm-omni$" $name -}}
+true
+{{- end -}}
+{{- end }}
+
 {{/*
 Workaround for models which don't yet contain chat templates in their HuggingFace repos.
 If a chat template is provided in the Helm values then this is used, otherwise we omit the
@@ -91,4 +102,4 @@ not responsive to adding a chat template to their repo.
 - --chat-template
 - {{ quote "{% for message in messages %}{% if message['role'] == 'system' %}{% endif %}{% if message['role'] == 'user' %}{{ '### Instruction:\n' }}{% endif %}{% if message['role'] == 'assistant' %}{{ '### Response:\n' }}{% endif %}{{ message['content'].strip() }}{% if not loop.last %}{{ '\n\n' }}{% endif %}{% if message['role'] == 'user' and loop.last %}{{ '### Response:\n' }}{% endif %}{% endfor %}" }}
 {{- end -}}
-{{- end }}
+{{- end }}

charts/azimuth-llm/templates/api/deployment.yml

@@ -26,7 +26,7 @@ spec:
         {{- else }}
         image: "{{ .Values.api.image.containerImage }}:{{ .Values.api.image.version }}"
         {{- end }}
-        {{- if eq .Values.api.image.containerImage "vllm/vllm-omni" }}
+        {{- if include "azimuth-llm.isOmniImage" . }}
         command:
         - vllm
         - serve
@@ -38,7 +38,7 @@ spec:
         - name: data
           mountPath: /root/.cache/huggingface
         args:
-          {{- if eq .Values.api.image.containerImage "vllm/vllm-omni" }}
+          {{- if include "azimuth-llm.isOmniImage" . }}
           - --omni
           {{- end }}
           {{- if semverCompare "<v0.13.0" .Values.api.image.version }}

charts/azimuth-omni-backend/.helmignore

@@ -0,0 +1,33 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
+
+# Others
+README.md
+Dockerfile
+*kubeconfig.y[a]ml
+venv/
+__pycache__/
+images/
+*.secret
+ci/

charts/azimuth-omni-backend/Chart.yaml

@@ -0,0 +1,16 @@
+apiVersion: v2
+name: azimuth-llm-omni-backend
+description: |
+  In-cluster vLLM backends for the Omni multimodal interface
+  (text-to-text / chat, text-to-speech, text-to-image).
+maintainers:
+  - name: "Victor HANG"
+    email: victor@stackhpc.com
+
+type: application
+
+# The version and appVersion are updated by the chart build script
+version: 0.1.0
+appVersion: local
+
+icon: https://huggingface.co/datasets/huggingface/brand-assets/resolve/main/hf-logo.svg

charts/azimuth-omni-backend/ci/test-values.yaml

@@ -0,0 +1,13 @@
+# CI: only exercise the TTT backend with the smallest possible model.
+ttt:
+  enabled: true
+  huggingface:
+    model: HuggingFaceTB/SmolLM2-135M-Instruct
+  api:
+    monitoring:
+      enabled: false
+    gpus: 1
+tts:
+  enabled: false
+tti:
+  enabled: false

charts/azimuth-omni-backend/templates/NOTES.txt

@@ -0,0 +1,17 @@
+Azimuth Omni backends provide one or more in-cluster vLLM model deployments
+(text-to-text / chat, text-to-speech, text-to-image) for the Omni interface.
+
+Enabled backends in this release:
+{{- range $alias := list "ttt" "tts" "tti" }}
+{{- $sub := index $.Values $alias }}
+{{- if and $sub $sub.enabled }}
+  - {{ $alias }}: {{ $sub.huggingface.model }} (in-cluster at http://{{ $.Release.Name }}-{{ $alias }}.{{ $.Release.Namespace }}.svc)
+{{- else }}
+  - {{ $alias }}: disabled (toggle with `{{ $alias }}.enabled=true`)
+{{- end }}
+{{- end }}
+
+Each enabled backend downloads its model weights from HuggingFace on first
+start, which can take a while.
+
+Release notes: https://github.com/stackhpc/azimuth-llm/releases