docs(blog): add post on streaming vLLM weights from Azure Blob Storage#5845
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new AKS blog post that walks through serving microsoft/phi-4 with vLLM while streaming weights directly from Azure Blob Storage via the RunAI Model Streamer (az://) using workload identity, plus a new author profile entry to support the post.
Changes:
- Added a new blog post: streaming vLLM weights from Azure Blob on AKS with workload identity.
- Added a new author key (
hariharan-sethuraman) towebsite/blog/authors.yml.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| website/blog/authors.yml | Adds a new author entry used by the new blog post. |
| website/blog/2026-06-26-runai-streamer-vllm/index.md | New end-to-end tutorial post for streaming vLLM weights from Azure Blob Storage on AKS. |
surajssd
force-pushed
the
add-run-ai-blog
branch
from
June 29, 2026 18:57
82c3a20 to
91c179e
Compare
surajssd
force-pushed
the
add-run-ai-blog
branch
from
June 30, 2026 00:20
3992664 to
9e03686
Compare
sdesai345
reviewed
Jun 30, 2026
sdesai345
reviewed
Jun 30, 2026
surajssd
force-pushed
the
add-run-ai-blog
branch
from
July 13, 2026 23:15
a814f49 to
6ed9d99
Compare
Add an AKS Engineering Blog post walking through serving `microsoft/phi-4`
with vLLM on AKS while streaming model weights directly from Azure Blob
Storage via the RunAI Model Streamer's native `az://` scheme, on a fully
managed A100 GPU node pool with workload identity (no storage keys).
- New post `website/blog/2026-07-13-runai-streamer-vllm/index.md` covering
cluster + managed GPU node pool (`--enable-managed-gpu=true`), premium
block-blob storage, keyless Blob access via workload identity, an
in-cluster upload `Job`, and a vLLM `Deployment` streaming with
`--load-format runai_streamer`
- Include a measured same-node benchmark (~187 s download-then-load vs
~10 s streamed; ~74 s vs ~10 s for the load step alone)
- Pull the `Job` and `Deployment`/`Service` manifests from
`Azure-Samples/aks-samples` via `curl ... | envsubst | kubectl apply`,
using a dedicated `runai-streamer` service account
- Add four diagrams with alt text and use Docusaurus admonitions
- Add author entry `hariharan-sethuraman` to `website/blog/authors.yml`
Signed-off-by: Suraj Deshmukh <suraj.deshmukh@microsoft.com>
surajssd
force-pushed
the
add-run-ai-blog
branch
from
July 14, 2026 00:20
0ccbafb to
81231ed
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new AKS Engineering Blog post, "Stream Model Weights to NVIDIA GPU (vLLM) from Azure Blob Storage using the RunAI Model Streamer" (
website/blog/2026-07-13-runai-streamer-vllm/index.md), a runnable end-to-end walkthrough for servingmicrosoft/phi-4with vLLM on AKS while streaming model weights directly from Azure Blob Storage via the RunAI Model Streamer's nativeaz://scheme. The post leans on a fully managed A100 GPU node pool and workload identity so no storage keys are needed, and explains — now with measured numbers — why streaming beats the default download-then-load path for autoscaling inference cold starts.Changes
index.md) walking through: deploying an AKS cluster with OIDC + workload identity and a managed GPU node pool (--enable-managed-gpu=true), creating a premium block-blob storage account, wiring up workload identity for keyless Blob access, an in-cluster uploadJobthat pushesmicrosoft/phi-4weights to Blob, and a vLLMDeploymentthat streams them via--load-format runai_streamer. Includes "Why stream", "Trade-offs and downsides", a verification step (spotting theLoading safetensors using Runai Model Streamerlog line), and a conclusion.1-why-stream-vs-downloaddiagram: a same-node (Standard_NC24ads_A100_v4), same-model comparison showing ~187 s for the default HuggingFace → local disk → GPU path versus ~10 s streaming from Blob withrunai_streamer— with the load-step-only figure (~74 s vs ~10 s, ~7×) called out as the stable, apples-to-apples number.JobandDeployment/ServiceYAML now live inAzure-Samples/aks-samplesand are pulled viacurl … | envsubst | kubectl apply, keeping the post copy-pastable while moving the manifests out of the prose (addresses review feedback that inline YAML would be hard to maintain).runai-streamerservice account applied declaratively fromservice-account.yamlinstead of mutating the namespacedefaultSA, so the workload-identity wiring stays off unrelated pods.until/sleeppolling loops replaced with "re-run this check" instructions.1-why-stream-vs-download.png,2-identity.png(workload-identity trust chain), and3-end-to-end.png(end-to-end flow), each with descriptive alt text.hariharan-sethuramanadded towebsite/blog/authors.yml; post co-authored withsuraj-deshmukh.<!-- truncate -->marker above the fold, refined title/description, applied grammar fixes, and corrected the workload-identity label explanation, thephi-4size figure, and depersonalized theConfigurationvariables.2026-07-13(front matterdate:and the post directory name kept in sync so the rendered date matches the/2026/07/13/runai-streamer-vllmURL).:::note/:::caution/:::tipadmonitions and replaced the Mermaid diagrams with images.Test Plan
npm run buildsucceeds inwebsite/(static site compiles; post renders at/2026/07/13/runai-streamer-vllmwith both authors).markdownlint-cli2passes againstblog/linters/.markdown-lint.yml(0 errors).codespellclean with the repo's ignore list.nvidia.com/gpu: 1with no GPU operator, the uploadJobpushed all ~27 GB ofmicrosoft/phi-4to Blob, vLLM (vllm/vllm-openai:v0.23.0) streamed fromaz://(confirmed via theLoading safetensors using Runai Model Streamerlog line), and a chat-completion request returned a valid response.