Repository navigation
Meter teacher requests through supported DSPy engines - #102
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Teacher request accounting on DSPy3.4 used the deprecated BaseLM.forward override. This moves the3.4 path to an owned supported engine, reserves before each provider attempt, observes raw identity/usage/cost before canonical conversion, and propagates owned typed budget/drift failures. The composition wrapper excludes complete_legacy so ordinary Predict calls cannot bypass metering. The supported3.3.1 legacy branch remains intact, including terminal identity refusal; the <3.5 cap stays.
Closes #93. Key/base/timeout overrides reach loopback, retries/cache/history controls stay off, and unknown cost remains null. Primary upstream and installed pinned source were rechecked.
Validation on isolated f9080cb: full suite628 passed, zero skipped on each DSPy3.3.1 and3.4.0; targeted17 passed each; independent code review passed; whole-tree Ruff passed; wheel/sdist built and50 package files/four entrypoints verified. Fresh installed-wheel flat/hierarchy no-key demos and runtime/helper lazy imports passed outside checkout with zero provider calls.
Retained exploratory failure: an early inherited complete_legacy hook bypassed the proposed engine and an unguarded synthetic test attempted an uncredentialed external request, returningHTTP401. The wrapper and hard loopback-only/socket guards now prevent that path; no paid inference or private data was authorized or executed. Current qualifying tests use guarded in-memory/loopback doubles. Exact-head hosted checks remain required before merge.