Migrate to PowerIO 0.9 (powerio C ABI 5) - #76
Open
samtalki wants to merge 3 commits into
Open
Conversation
Track PowerIO 0.9. The binding gates its C ABI handshake on equality, so the package and its binaries move together and the `[compat]` bound cannot be relaxed below 0.9. Three things in 0.9 reach PowerDiff, and each is answered where it belongs. The format vocabulary is PowerIO's, not a copy of it. `parse_file` maintained a five-format allowlist that refused everything else, so every reader PowerIO shipped needed a PowerDiff release to become reachable and the error message named a set that was already wrong. An unrecognized `from` token is now handed to PowerIO, which answers with what the linked library actually reads. PowerDiff's own short spellings still resolve, a bare `json` is still refused as ambiguous, and a distribution token is refused with the lowering step to run first. The normalize pass is run here rather than inside `to_powerdata`, which recognizes an already-normalized input and skips its own — one pass, not two. 0.9 raises that pass's fidelity findings as a `@warn` burst from inside the bridge, once per constructor and with nothing saying whose they are; running the pass here returns them as data instead. New exported `network_findings(net)` gives both channels, reader and normalize, as `CODE: message` lines. PowerDiff reports the normalize findings itself, one per distinct code, once per network, under `silence()`. The ingest is memoized on the parsed network, so `DCNetwork` and `ACNetwork` built from one network share it and cannot describe different cases. Generator reactive limits may now be absent. 0.9 carries a bound the case does not state as `±Inf` rather than refusing the case, and stock case9241pegase leaves them off seven generators. The KKT layout is fixed, one complementarity row per bound, so the bound can be neither dropped nor carried: `ρ · (qg - qmin)` with `qmin == -Inf` is `0 * Inf`, a NaN in the residual and an Inf in the Jacobian. The bound is left off the solver model and the row reads `ρ = 0` — the multiplier of a constraint that is not there, and what a solver reports for a bound it was never given, so the solved duals and the residual agree. Its derivative is constant, so the fixed-regime sensitivity is zero. Sparsity is unchanged. Without this the upgrade would have turned a clean refusal into a silent NaN. A non-finite value anywhere else is a modeling error, now named with its element and field instead of reaching a factorization; `ACNetwork` applies the same rule to a caller-built table. A branch whose rating is non-finite is unrated and takes the same synthesized thermal limit as `rate_a == 0`. `PowerIO.source_format` reports the lowercase token every `from` accepts (`"powermodels-json"`, not `"PowerModelsJson"`); the parser tests assert it. test/test_powerio_integration.jl covers the seam: format routing, the one-pass memoized ingest, findings, absent bounds, and the values PowerDiff refuses. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018zCNDAiTv7ZjZKbBLFM9EM
The ingest is only valid for the live Rust handle it was read through, so `__init__` empties it: a handle does not survive a process, and a precompile workload added later must not be able to carry an entry into a session. The distribution-format refusal named `lower_multiconductor_to_balanced` alone, which takes a `NetworkPackage` rather than a `MulticonductorNetwork`. Name the whole route: `to_package`, lower, `from_package`. Assert `Diagnostic` element-wise rather than on the container's element type, which an empty findings vector need not carry. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018zCNDAiTv7ZjZKbBLFM9EM
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018zCNDAiTv7ZjZKbBLFM9EM
Contributor
Benchmark Results (Julia v1)Time benchmarks
Memory benchmarks
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Track PowerIO 0.9. The binding gates its C ABI handshake on equality, so the package and its binaries move together and
[compat] PowerIO = "0.9"cannot be relaxed below it.Three things in 0.9 reach PowerDiff, and each is answered where it belongs.
The format vocabulary is PowerIO's, not a copy of it
parse_filemaintained a five-format allowlist and refused everything else, so every reader PowerIO shipped needed a PowerDiff release to become reachable, and the error message named a set that was already wrong. An unrecognizedfromtoken now goes to PowerIO, which answers with what the linked library actually reads.Reachable as a result, with no code here naming them:
pandapower,pypsa,pslf/epc,pwb,gridfm,goc3,surge,opfdata,psse34,psse35.PowerDiff's own short spellings (
:m,:raw,:aux,:pm,:powermodels,:egret) still resolve. Two answers stay local because PowerIO cannot give them: a barejsonnames a container rather than a reader, and a distribution token parses to aMulticonductorNetwork, refused with the lowering route to run first.One normalize pass, findings as data
0.9 raises the normalize pass's fidelity findings as a
@warnburst from insideto_powerdata— once per constructor call, with nothing saying whose they are or which network they describe.PowerDiff runs that pass itself now and reads
to_powerdataoff the normalized network, which recognizes an already-normalized input and skips its own. One pass, not two, with the findings returned as values instead of logged.New exported
network_findings(net)returns both channels —reader(what the source format could not represent) andnormalize— asCODE: messagelines. PowerDiff reports the normalize findings itself, one per distinct code, once per network, undersilence().The ingest is memoized on the parsed network, so
DCNetwork(net)andACNetwork(net)share it: the JSON payload materializes once, each finding is said once, and the two constructors can no longer describe different cases.Generator reactive limits may be absent
0.9 carries a bound the case does not state as
±Infrather than refusing the case. That is how MATPOWER, PowerModels, pandapower and PyPSA all spell "no limit", and stock case9241pegase leaves the reactive limits off seven generators.The KKT layout is fixed, one complementarity row per bound, so the bound can be neither dropped nor carried:
ρ · (qg − qmin)withqmin == -Infis0 * Inf— aNaNin the residual and anInfin the Jacobian.So the bound is left off the solver model, and its row reads
ρ = 0: the multiplier of a constraint that is not there, and what a solver reports for a bound it was never given, so the solved duals and the residual agree by construction. Its derivative is constant, so the fixed-regime sensitivity of an absent bound is zero. Sparsity is unchanged — a zero entry is dropped exactly as a non-binding stated bound's already is.Without this the upgrade would have turned a clean refusal into a silent NaN, on stock pglib cases.
A non-finite value anywhere else is a modeling error, now named with its element and field instead of reaching a factorization;
ACNetworkapplies the same rule to a caller-built table. A branch whose rating is non-finite is unrated and takes the same synthesized thermal limit asrate_a == 0.Also
PowerIO.source_formatreports the lowercase token everyfromargument accepts ("powermodels-json", not"PowerModelsJson") — a powerio 0.9 break. The parser tests assert the new spelling.__init__clears the ingest cache: an ingest is only valid for the live Rust handle it was read through, so none may reach a session from a precompiled image.Verification status
CI is green on
e831561—test,build(docs, with doctests and cross-reference checks), andbenchmarkall pass.test-fullandtest-nightlyare skipped onpull_requestby design.This PR was authored in an environment with no Julia toolchain — both
julialang-s3.julialang.organd GitHub release assets are refused by its network policy — and CI does not run on branch pushes, so opening the PR triggered the first execution of any of it. Everything below the diff was reviewed by hand first; the run is what confirms it.The new testsets all executed and passed:
The diagnostics path is visible in the log doing its job on a real case: the PSS/E round-trip in
test_non_matpower_parsers.jlloses generator cost data, and each of the two networks it parses reportsPowerIO normalize: CANONICALIZE.NORMALIZE.GEN_COST_ABSENT: the case has 5 in-service generator(s) and no cost dataexactly once — one report per network, which is the dedupe contract.Benchmarks show no regression on PGLib IEEE 300;
dc_opf/kkt_jacobian/susceptanceis ~1.19× faster and the rest are within noise.Tests
test/test_powerio_integration.jlcovers the seam directly: format routing (aliases, passthrough, the two local refusals), the one-pass memoized ingest,network_findingson both channels, absent reactive limits end to end (solver model, solved duals, KKT residual, Jacobian entries), and the values PowerDiff refuses.Existing coverage carries the rest:
test_parser_parity.jlholds path and IO parsing to identical tables, andtest_non_matpower_parsers.jlround-trips MATPOWER through PowerModels JSON, Egret JSON and PSS/E RAW.Upstream
Four gaps found while doing this are filed rather than worked around here:
@warn-ing them (this PR pre-normalizes to work around it)to_powerdatarows carry nosource_row, so_active_source_rowsstill re-derives the filter on a second full extractionInfis accepted on every field, not just bounds (this PR checks each field itself)with C ABI companions at eigenergy/powerio#399 and eigenergy/powerio#400.
🤖 Generated with Claude Code
https://claude.ai/code/session_018zCNDAiTv7ZjZKbBLFM9EM