Skip to content

feat(asr): prefetch the native Distil-Whisper presets with docling-tools - #4825

Merged
PeterStaar-IBM merged 1 commit into
docling-project:mainfrom
BBC-Esq:feat/asr-distil-prefetch
Oct 16, 2026
Merged

PeterStaar-IBM merged 1 commit into
docling-project:mainfrom
BBC-Esq:feat/asr-distil-prefetch

Conversation

@BBC-Esq

@BBC-Esq BBC-Esq commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Note

Description of this pull request

This PR adds the four native Distil-Whisper presets to docling-tools models download, the same way #4760 added the native Whisper checkpoints:

docling-tools models download whisper_distil_small_en --output-dir /models
docling --asr-model whisper_distil_small_en_native --artifacts-path /models audio.wav

The new names are whisper_distil_small_en, whisper_distil_medium_en, whisper_distil_large_v3 and whisper_distil_large_v3_5. Each one downloads the OpenAI-format checkpoint that its preset loads, into the output directory in the Hugging Face cache layout. That is the layout the native transcriber already reads when artifacts_path is set, because it calls hf_hub_download(..., cache_dir=artifacts_path, local_files_only=True). Plain --all still excludes ASR models, and --all --asr now includes these four.

_DISTIL_WHISPER_OPENAI_CHECKPOINTS moves from asr_transcriber.py to asr_model_specs.py, next to the presets it describes, so the downloader and the transcriber share one table. docling.utils may not import from docling.pipeline, and it already depends on vlm_model_specs for the same reason, so tach.toml gets the matching asr_model_specs edges.

When artifacts_path does not contain the checkpoint, the error message now gives the docling-tools command instead of hf download.

I tested it end to end. I prefetched distil-small.en with the new command into an empty folder, then loaded it with HF_HUB_OFFLINE=1 and transcribed a 94-second clip correctly. An empty folder still raises the FileNotFoundError, and the message now names the command that fixes it.

Important

Related pull requests: step 2 of 4

These four pull requests add offline support (--artifacts-path and docling-tools models download) for the ASR presets I contributed in #2938 and #3741, the same way that Peter Staar did in #4760 for native Whisper. A checked box means that step is merged.

  • Cleanup (refactor(asr): remove redundant n_mels override for WhisperS2T #4785): remove the redundant n_mels override in the WhisperS2T backend.
  • PR A: add the native Distil-Whisper presets to docling-tools models download. ⬅️ this PR
  • whisper-s2t-reborn 1.8.0: let the library load its models, including the word aligner, from a local folder.
  • PR B: run the WhisperS2T presets offline from --artifacts-path, with prefetch through docling-tools models download.

Related work for another backend: #4771 aims at the same goal for a new Transformers ASR backend (with MLX presets): its presets run Transformers-format checkpoints offline from --artifacts-path (#4684). Both PRs change neighboring lines in asr_transcriber.py, so the second one to merge needs a small merge; if that is this PR, I will resolve it here.

Checklist:

  • Documentation has been updated, if necessary.
  • Examples have been added, if necessary.
  • Tests have been added, if necessary.

Add whisper_distil_small_en, whisper_distil_medium_en, whisper_distil_large_v3 and whisper_distil_large_v3_5 to docling-tools models download. They fetch the OpenAI-format checkpoints that the native Distil presets load, into the output directory's Hugging Face cache layout, which is the layout the native transcriber already reads when artifacts_path is set. Plain --all still excludes them, and --all --asr includes them.

Move _DISTIL_WHISPER_OPENAI_CHECKPOINTS from the transcriber to asr_model_specs, next to the presets it describes, so the downloader and the transcriber share one table. docling.utils already depends on vlm_model_specs for the same reason, so tach.toml gets the matching asr_model_specs edges.

The missing-checkpoint error under artifacts_path now gives the docling-tools command instead of hf download.

Signed-off-by: BBC, Esquire <bbc@chintellalaw.com>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @BBC-Esq, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 14, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-needs-care AI triage: check CI-safety findings before you approve CI asr Issues related to ASR (Automatic Speech Recognition) CLI issues related to the command line interface (CLI) enhancement New feature or request labels Oct 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 65bd8dec2e. The analysis read the diff only. It did not run code from this pull request.

This pull request adds native Distil-Whisper model presets to docling-tools models download. It moves checkpoint metadata to docling.datamodel.asr_model_specs and updates CLI tests.

Topic labels added: enhancement, asr, CLI.

CI safety: needs maintainer care before you approve CI.

  • tach.toml: changes CI test selection
3 related pull request(s) in the same code
PR State Signal Assessment
#4771 open changes the same code related: PR #4771 adds Transformers and MLX presets for ASR pipelines.
#4785 merged referenced as #4785 related: PR #4785 refactors options for WhisperS2T in the ASR transcriber.
#4760 merged referenced as #4760 related: PR #4760 added download support for standard native Whisper models to docling-tools.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI first review (advisory): no blocker found

This review covers the full diff at 65bd8dec2e. It read the code and did not run it.

The PR adds four Distil-Whisper presets to docling-tools models download, moves _DISTIL_WHISPER_OPENAI_CHECKPOINTS to asr_model_specs.py, updates the error message, and extends tach.toml. All four checkpoint keys round-trip correctly through _distil_whisper_prefetch_name to the enum values in _AvailableModels. The downloader uses hf_hub_download(cache_dir=output_dir) — the same layout the transcriber reads offline — which differs from the flat-file layout of regular Whisper models; this is intentional and consistent. No behavioral issues found; the human reviewer should confirm that PR #4771 (open, changes the same files) does not conflict with this one before merging.

A maintainer makes the merge decision. This review never approves the pull request.

@github-actions github-actions Bot added the ai:review-lgtm AI first review found no blocker or major issue (advisory) label Oct 14, 2026

@PeterStaar-IBM PeterStaar-IBM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@codecov

codecov Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 48.00000% with 13 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
docling/utils/model_downloader.py 13.33% 12 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@PeterStaar-IBM PeterStaar-IBM left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@PeterStaar-IBM
PeterStaar-IBM merged commit 483cf13 into docling-project:main Oct 16, 2026
53 checks passed
@BBC-Esq
BBC-Esq deleted the feat/asr-distil-prefetch branch October 16, 2026 12:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-needs-care AI triage: check CI-safety findings before you approve CI ai:review-lgtm AI first review found no blocker or major issue (advisory) asr Issues related to ASR (Automatic Speech Recognition) CLI issues related to the command line interface (CLI) enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants