Skip to content

fix: Bound embedding input length and attention memory per batch - #837

Draft
buildingjoshbetter wants to merge 2 commits into
maintenance/reranker-token-budget-v1from
maintenance/embed-token-budget-v1
Draft

fix: Bound embedding input length and attention memory per batch#837
buildingjoshbetter wants to merge 2 commits into
maintenance/reranker-token-budget-v1from
maintenance/embed-token-budget-v1

perf: Skip embedding budget measurement when a batch always fits

aa37923
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar