A calculus domain graded by two derivatives that must agree - #36
Merged
Conversation
… physics The battery gains calculus -- derivatives at a point through chain-rule compositions, exact definite integrals, tangent slopes, and quadratics built FROM their roots so the truth exists by construction. The truth for every question comes from mpeqs.calculus, and the domain tests grade each question through BOTH of its derivative paths -- symbolic and dual-number autograd -- so no answer rests on either implementation alone. Physics gains free fall, acceleration and kinetic energy, formula always stated in the question, heights BUILT from the answer so three decimals state them exactly. The %g trap struck again on the way -- six significant digits turned 3575.745 into "3575.74", a question whose stated figure disagreed with its graded answer -- and the round-trip test caught it before any model saw it. The first live run caught the GRADER being wrong twice, which is the better direction: * a model answered -42.666667 for -128/3 -- MORE precise than the grader's 4-decimal ceiling -- and was marked wrong for it. matches() now accepts 6 down to 2 decimal places: an answer must never fail for exceeding the grader's precision. * light_travel never asked for decimals, so "182 seconds" against 182.13 was the model rounding reasonably and the question being underspecified. It now asks for two decimal places. Measured live against qwen3 after the fixes: 19/22 (86%), seven of eleven groups perfect, pinned. The remaining misses are genuine model errors -- a dropped chain factor and a forgotten square root -- of exactly the kind the reasoning loop's machinery exists to catch when these questions arrive through it rather than bare. mpeqs floor raised to 0.4.0, verified by grep after the edit, because a silent replace-miss on this exact line has happened before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A calculus domain graded by two derivatives that must agree, and more physics
The battery gains calculus -- derivatives at a point through chain-rule
compositions, exact definite integrals, tangent slopes, and quadratics built
FROM their roots so the truth exists by construction. The truth for every
question comes from mpeqs.calculus, and the domain tests grade each question
through BOTH of its derivative paths -- symbolic and dual-number autograd -- so
no answer rests on either implementation alone.
Physics gains free fall, acceleration and kinetic energy, formula always stated
in the question, heights BUILT from the answer so three decimals state them
exactly. The %g trap struck again on the way -- six significant digits turned
3575.745 into "3575.74", a question whose stated figure disagreed with its
graded answer -- and the round-trip test caught it before any model saw it.
The first live run caught the GRADER being wrong twice, which is the better
direction:
4-decimal ceiling -- and was marked wrong for it. matches() now accepts 6
down to 2 decimal places: an answer must never fail for exceeding the
grader's precision.
the model rounding reasonably and the question being underspecified. It now
asks for two decimal places.
Measured live against qwen3 after the fixes: 19/22 (86%), seven of eleven
groups perfect, pinned. The remaining misses are genuine model errors -- a
dropped chain factor and a forgotten square root -- of exactly the kind the
reasoning loop's machinery exists to catch when these questions arrive through
it rather than bare.
mpeqs floor raised to 0.4.0, verified by grep after the edit, because a silent
replace-miss on this exact line has happened before.