Skip to content

Speed up Term.__init__, add shortcut path for Term() and Term(x) - #1283

Open
Zeroto521 wants to merge 13 commits into
scipopt:masterfrom
Zeroto521:expr/term-perf
Open

Zeroto521 wants to merge 13 commits into
scipopt:masterfrom
Zeroto521:expr/term-perf

Conversation

@Zeroto521

@Zeroto521 Zeroto521 commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

Term is created for every variable that enters an expression (x + y, 2 * x, quicksum(...), ...) and it is the key of the dict behind Expr.terms, so its constructor, its comparison and the * used to multiply monomials are all on the hot path of modelling code.

This PR adds a shortcut path for Const=Term() (n=0) and Term(Variable) (n=1), inspared by #1273. Terms with two or more variables construct through the unchanged code path, User can't call Term(x, y) directly in real code. Term(x, y) will be created from Term.__mul__.

case base(ns) head(ns) speedup user can call or not
Term() 315.7 64.6 4.89x YES (via CONST)
Term(v) 564.1 107.3 5.26x YES (via Variable)
Term(v, w) 724.6 610.9 1.19x NO (Fake Case)
Term(w, v) 658.3 691.7 0.95x NO (Fake Case)
Term(v, w, x) 772.2 805.0 0.96x NO (Fake Case)
Term(v, w, x, y) 951.9 905.4 1.05x NO (Fake Case)
Term(v) == Term(v) 115.3 36.7 3.14x YES
Term(v, w, x) == Term(v, w, x) 236.6 48.6 4.87x YES
dict[Term(v)] 111.2 44.3 2.51x YES
Term(v) * Term(w) 1799.5 385.9 4.66x YES
Term(v, w) * Term(x, y) 637.2 237.7 2.68x YES
(v + w) * (x + y) 3072.3 1757.2 1.75x YES
Term._evaluate (1 var) 58.9 54.8 1.07x YES
Term._evaluate (3 vars) 76.7 77.7 0.99x YES

Avoid sorted, the key lambda, and the generator used to build the hash for the common single-variable case by storing the fresh argument tuple directly and hashing the SCIP variable pointer. Handle the constant term explicitly as well.
Expose Variable.ptr() and Variable.getIndex() as cpdef so Cython callers avoid Python dispatch, and reuse them in Term instead of reading scip_var directly.

In the single-variable Term path, fetch the variable with PyTuple_GET_ITEM (the *args tuple is exact, so no bounds check is needed) and keep the Variable type check through the <object> cast, so non-Variable arguments still raise TypeError instead of being read as SCIP_VAR*.
Term.vartuple only ever holds Variables (guaranteed by __init__ and __mul__), so the unchecked PyTuple_GET_ITEM is safe in _evaluate and matches what __eq__ and __mul__ already do.

Take the lengths from len(self.vartuple) instead of len(self), which is a PyObject_Length call dispatching through Term.__len__, and use Py_ssize_t for the lengths and loop counters: list/tuple sizes are Py_ssize_t, and storing them in an int would truncate silently.
Comment thread src/pyscipopt/scip.pxi Resolved
@Joao-Dionisio
Joao-Dionisio self-requested a review October 29, 2026 15:28
hash(tuple(v.ptr() for v in res.vartuple)) goes through a generator and calls the Python-level Variable.ptr() once per variable. Collect the pointers in the merge loop instead and hash the resulting tuple, which keeps the hash identical to the one Term.__init__ builds for the same variables.
Comment thread src/pyscipopt/expr.pxi
self.vartuple = vartuple
self.hashval = <Py_ssize_t>hash((var.ptr(),))

else:

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should Term forbid n > 1? Term(x, y) (n>1) cannot be initialized directly, it’s always comes from Term.mul

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But users could still use it to access the coefficients of a quadratic constraint, for example, right?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about this, @Zeroto521 ? It's not a big deal either way for me

@Zeroto521 Zeroto521 Nov 22, 2026 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer Term should receive one single variable or a None.
The normal user would call x^3 instead of Term(x, x, x) and they don’t need to know how to import Term. Term is kind of an inner class.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, we can change this. But I think there's a test that'd need to change as well.

@Joao-Dionisio Joao-Dionisio added this to the v6.4.0 milestone Nov 15, 2026
@Joao-Dionisio

Copy link
Copy Markdown
Member

Hey @Zeroto521 , as always sorry for taking so long! I will try to have this reviewed properly by the end of next weekend.

@Joao-Dionisio

Copy link
Copy Markdown
Member

/benchmark

@github-actions

github-actions Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

✅ /benchmark by @Joao-Dionisio: results

@github-actions

Copy link
Copy Markdown
Contributor

Results for /benchmark by @Joao-Dionisio

Benchmark results (base → PR)

No regressions

Benchmark Instructions Time Peak memory Retained memory
test_evaluate_nonlinear 4.247 G → 3.915 G (-7.8%) 🟢 176 B → 176 B (+0.0%) 0 B → 0 B
test_expr_arithmetic 5.175 G → 5.175 G (+0.0%) 1.2 MB → 1.2 MB (+0.0%) 32 B → 32 B (+0.0%)
test_expr_ops 6.221 G → 4.777 G (-23.2%) 🟢 137.7 kB → 25.7 kB (-81.3%) 🟢 96 B → 32 B (-66.7%)
test_matrix_ops 6.397 G → 6.397 G (+0.0%) 71.5 MB → 71.5 MB (+0.0%) 1.3 kB → 1.3 kB (+0.0%)
test_nonlinear_expr 3.150 G → 3.067 G (-2.7%) 🟢 11.1 MB → 11.1 MB (+0.0%) 32 B → 32 B (+0.0%)
test_quadratic_expr 6.874 G → 5.224 G (-24.0%) 🟢 89.0 MB → 89.0 MB (-0.0%) 96 B → 32 B (-66.7%)
test_quicksum 4.199 G → 4.198 G (-0.0%) 10.0 MB → 10.0 MB (+0.0%) 32 B → 32 B (+0.0%)
test_repeated_terms 5.767 G → 5.774 G (+0.1%) 9.8 kB → 9.8 kB (+0.0%) 32 B → 32 B (+0.0%)
test_add_cons_linear 5.015 G → 5.024 G (+0.2%) 36.9 MB → 36.9 MB (+0.0%) 36.9 MB → 36.9 MB (+0.0%)
test_add_linear_conss 18.44 s → 18.75 s (+1.7%) 259.9 MB → 259.9 MB (+0.0%) 259.9 MB → 259.9 MB (+0.0%)
test_add_nonlinear_conss 5.894 G → 5.786 G (-1.8%) 🟢 2.5 MB → 2.4 MB (-4.5%) 2.4 MB → 2.4 MB (-0.0%)
test_add_vars 2.996 G → 2.137 G (-28.7%) 🟢 47.6 MB → 47.6 MB (-0.0%) 32 B → 32 B (+0.0%)
test_enum_getters 1.031 G → 1.031 G (+0.0%) 184 B → 184 B (+0.0%) 0 B → 0 B
test_getters_after_solve 5.473 G → 5.158 G (-5.8%) 🟢 176 B → 176 B (+0.0%) 0 B → 0 B
test_plugin_callbacks 3.375 G → 3.376 G (+0.0%) 16.0 kB → 15.9 kB (-0.3%) 15.0 kB → 15.0 kB (+0.0%)
test_branch_and_price_columns 9.73 s → 9.58 s (-1.5%) 95.4 MB → 95.4 MB (-0.0%) 95.4 MB → 95.4 MB (+0.0%)
test_many_small_models 8.684 G → 8.606 G (-0.9%) 1.0 MB → 931.7 kB (-8.4%) 104 B → 104 B (+0.0%)
test_modify_model 7.555 G → 7.417 G (-1.8%) 🟢 9.7 MB → 9.7 MB (+0.0%) 589.9 kB → 589.9 kB (+0.0%)
test_resolve_cycle 1.737 G → 1.705 G (-1.8%) 🟢 2.8 MB → 2.8 MB (+0.0%) 590.2 kB → 590.2 kB (+0.0%)

Instructions from callgrind, time is the best of several runs, memory is Python-side (tracemalloc). Flagged above 1% instructions, 20% time, 5% peak memory, 5% retained memory.

Against v6.3.0

v6.3.0 → PR

⚠️ 1 regression

Benchmark Instructions Time Peak memory Retained memory
test_evaluate_nonlinear 4.474 G → 3.915 G (-12.5%) 🟢 200 B → 176 B (-12.0%) 0 B → 0 B
test_expr_arithmetic 5.175 G → 5.175 G (+0.0%) 1.2 MB → 1.2 MB (+0.0%) 32 B → 32 B (+0.0%)
test_expr_ops 6.225 G → 4.777 G (-23.3%) 🟢 137.7 kB → 25.7 kB (-81.3%) 🟢 96 B → 32 B (-66.7%)
test_matrix_ops 6.394 G → 6.397 G (+0.0%) 71.5 MB → 71.5 MB (+0.0%) 1.3 kB → 1.3 kB (+0.0%)
test_nonlinear_expr 3.148 G → 3.067 G (-2.6%) 🟢 12.2 MB → 11.1 MB (-9.2%) 🟢 32 B → 32 B (+0.0%)
test_quadratic_expr 6.872 G → 5.224 G (-24.0%) 🟢 89.0 MB → 89.0 MB (-0.0%) 96 B → 32 B (-66.7%)
test_quicksum 4.199 G → 4.198 G (-0.0%) 10.0 MB → 10.0 MB (+0.0%) 32 B → 32 B (+0.0%)
test_repeated_terms 5.780 G → 5.774 G (-0.1%) 9.8 kB → 9.8 kB (+0.0%) 32 B → 32 B (+0.0%)
test_add_cons_linear - → 5.024 G - → 36.9 MB - → 36.9 MB
test_add_linear_conss 18.62 s → 18.75 s (+0.7%) 259.9 MB → 259.9 MB (+0.0%) 259.9 MB → 259.9 MB (+0.0%)
test_add_nonlinear_conss 6.784 G → 5.786 G (-14.7%) 🟢 2.5 MB → 2.4 MB (-4.6%) 2.4 MB → 2.4 MB (-0.0%)
test_add_vars 2.999 G → 2.137 G (-28.7%) 🟢 47.6 MB → 47.6 MB (-0.0%) 32 B → 32 B (+0.0%)
test_enum_getters 593.125 M → 1.031 G (+73.8%) 🔴 184 B → 184 B (+0.0%) 0 B → 0 B
test_getters_after_solve 5.323 G → 5.158 G (-3.1%) 🟢 176 B → 176 B (+0.0%) 0 B → 0 B
test_plugin_callbacks 3.352 G → 3.376 G (+0.7%) 16.0 kB → 15.9 kB (-0.3%) 15.0 kB → 15.0 kB (+0.0%)
test_branch_and_price_columns 9.96 s → 9.58 s (-3.7%) 95.4 MB → 95.4 MB (-0.0%) 95.4 MB → 95.4 MB (+0.0%)
test_many_small_models 8.689 G → 8.606 G (-1.0%) 1.0 MB → 931.7 kB (-8.4%) 104 B → 104 B (+0.0%)
test_modify_model 7.581 G → 7.417 G (-2.2%) 🟢 9.7 MB → 9.7 MB (+0.0%) 589.9 kB → 589.9 kB (+0.0%)
test_resolve_cycle 1.730 G → 1.705 G (-1.4%) 🟢 2.8 MB → 2.8 MB (+0.0%) 590.2 kB → 590.2 kB (+0.0%)

Instructions from callgrind, time is the best of several runs, memory is Python-side (tracemalloc). Flagged above 1% instructions, 20% time, 5% peak memory, 5% retained memory.

Base f65daa985dd0b41a1e65f12b3b645148828c86d4, PR 8598bd4ef1c5387885751003245212c5637499d9. Run

@Joao-Dionisio

Copy link
Copy Markdown
Member

The following used to return an error, but now segfaults:

  • Term(None)
  • Term(x) * None
  • m.delVar(None)

Comment thread src/pyscipopt/expr.pxi Resolved
Comment thread src/pyscipopt/expr.pxi Resolved
Zeroto521 and others added 4 commits November 21, 2026 10:21
Co-authored-by: João Dionísio <57299939+Joao-Dionisio@users.noreply.github.com>
Co-authored-by: João Dionísio <57299939+Joao-Dionisio@users.noreply.github.com>
The *args items are not type-checked by Cython, and assigning None to a
cdef class variable is allowed (it stores Py_None), so Term(None) stored
Py_None in a Variable variable and crashed in var.ptr(), which dispatches
through the vtab pointer read from the Py_None object.

Validate the arguments up front and raise TypeError, matching the error
already raised for other non-Variable arguments. The single-variable fast
path can then use an unchecked cast.

Add a regression test covering None and non-Variable arguments.
Comment thread src/pyscipopt/expr.pxi
Comment on lines 81 to +91
def __init__(self, *vartuple: Variable):
self.vartuple = tuple(sorted(vartuple, key=lambda v: v.getIndex()))
self.hashval = <Py_ssize_t>hash(tuple(v.ptr() for v in self.vartuple))
cdef Py_ssize_t i, n = PyTuple_GET_SIZE(vartuple)
cdef Variable var
cdef object item

for i in range(n):
item = <object>PyTuple_GET_ITEM(vartuple, i)
if not isinstance(item, Variable):
raise TypeError(
f"Term() arguments must be Variable, got {type(item).__name__}"
)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The following used to return an error, but now segfaults:

  • Term(None)
  • Term(x) * None
  • m.delVar(None)

fixed

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants