Skip to content

Restore the v3 public names as a permanent v4 API - #300

Merged
jtdub merged 4 commits into
nextfrom
restore-v3-names-in-v4
Aug 27, 2026
Merged

Restore the v3 public names as a permanent v4 API#300
jtdub merged 4 commits into
nextfrom
restore-v3-names-in-v4

Conversation

@jtdub

@jtdub jtdub commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Makes v4 a non-breaking upgrade for v3 users. Every v3 public name that v4 renamed or removed is restored as a thin delegation to its v4 counterpart — no DeprecationWarning, no planned removal. The old spellings become a supported, documented, tested part of the v4 API.

nautobot-golden-config drove the scope. Its whole hier_config surface is six items; three were missing and blocked it entirely (get_hconfig, hconfig_v2_os_v3_platform_mapper, load_hconfig_v2_options). Its code now runs unedited once it widens the hier-config = "^3.4.1" pin.

What is restored

Area Names
Constructors get_hconfig, get_hconfig_fast_load, get_hconfig_from_dump, get_hconfig_fast_generic_load; get_hconfig_driver re-exported from hier_config.constructors
Methods config_to_get_to, dump_simple, cisco_style_text, tags_add, tags_remove, use_default_for_negation
Utils load_hconfig_v2_options, load_hconfig_v2_tags, load_hconfig_v2_options_from_file, HCONFIG_PLATFORM_V2_TO_V3_MAPPING, both platform mappers
Models NegationDefaultWithRule, NegationDefaultWhenRule, NegationSubRule

Delegating rather than copying means the two spellings cannot drift apart.

Negation rules stay live

HConfigDriverRules keeps negate_with, negation_default_when, and negation_sub as declared fields, so mypy and pyright accept them. all_negation_rules() resolves them together with the unified negation list on every lookup, rather than folding once at validation.

That matters: v3's own docs/user/custom-drivers.md teaches driver.rules.negate_with.append(...) in three places. Folding at validation left that idiom running silently with no effect, producing a different — potentially destructive — remediation. Resolving at lookup time also means re-validating a rules model never duplicates rules, so HConfigDriverRules(**base.model_dump(), negate_with=[...]) is safe. The no-v3-rules path returns negation itself, so the common case allocates nothing.

Proving parity, not asserting it

tests/integration/v3_scenarios.py mirrors nautobot_golden_config.models._get_hierconfig_remediation step for step and imports under both major versions. It covers 29 scenarios: the plain golden config path over three fixture pairs, every driver name the v2 mapper knows plus the unknown and whitespace-padded forms, a full v2 options dict firing all three negation strategies, the same rules applied by append, tag filtering, and the restored names golden config does not call.

Two tests consume it:

  • test_v3_baseline.py compares against tests/fixtures/v3_baseline.json, a committed recording made on 3.7.0. Runs in the normal suite — no network, no second environment.
  • test_v3_differential.py diffs against a live v3 install. Deselected by default via the new v3_differential marker; run it with poetry run pytest -m v3_differential.

All 29 scenarios match v3.7.0 byte for byte. Regenerate the recording with ./scripts/generate_v3_baseline.py.

Still breaking, and documented as such

Three v4 changes cannot be absorbed by a name alias. docs/user/v3-compatibility.md and the migration guide both state them:

  • depth() became the depth property — the one required call-site edit.
  • DriverNotFoundError / InvalidConfigError / IncompatibleDriverError replace ValueError.
  • The EOS, NX-OS, and XR views return real data where v3 raised NotImplementedError.

So 4.0.0 remains correct under semver.

Also in here

  • New page docs/user/v3-compatibility.md, added to the nav.
  • The migration guide reframes the renames as recommended rather than required.
  • CHANGELOG.md's ### Removed section listed names this branch retains. Corrected.

Known gap, deliberately left out

HCONFIG_PLATFORM_V2_TO_V3_MAPPING has no entries for HP_PROCURVE, HP_COMWARE5, or FORTINET_FORTIOS, so those devices fall through to Platform.GENERIC and get a wrong remediation with no error. The table is restored verbatim from v3, so adding entries would change behavior and break the v3 parity contract these tests enforce. It also needs netutils' exact driver-name strings, which I did not want to guess. Worth a follow-up issue.

Self-Review Checklist

  • poetry run ./scripts/build.py lint-and-test passes locally (lint + 95% test coverage). 846 tests, 97% coverage.
  • Tests were written first (TDD) and cover the change, following the testing conventions.
  • CHANGELOG.md has an entry under ## [Unreleased] referencing this issue/PR ((#NNN)).
  • Documentation is updated if public API or driver behavior changed (and mkdocs build --strict passes if docs were touched).
  • Commit messages follow the contributing guide: imperative mood, subject ≤72 characters, body explains why.

AI-Assisted Contributions

Written with Claude Code. Reviewed with /code-review and /security-review before opening.

The code review found one HIGH issue — the negation append idiom described above — which is fixed in 2086635, along with a non-idempotent validator, the missing use_default_for_negation, an overstated changelog, and a wrong doctest.

The security review found no vulnerabilities. It cleared formats.py XML parsing (no XXE), all YAML loading (safe_load throughout), the driver registry (no import-by-string), and both subprocess.run calls (list argv, no interpolated data).

https://claude.ai/code/session_01HbMucj2WqnL685BRUcFhZT

jtdub added 2 commits August 27, 2026 15:59
v4 renamed or removed a set of names that v3 users call every day, with no
alias anywhere in the package. A v3 consumer upgrading hit an ImportError or
AttributeError on the first line. nautobot-golden-config is the reference
case: three missing names (get_hconfig, hconfig_v2_os_v3_platform_mapper,
load_hconfig_v2_options) blocked it entirely.

Restore every renamed or removed name as a thin delegation to its v4
counterpart. No DeprecationWarning, no planned removal -- the old spellings
are a supported part of the v4 API. Delegating rather than copying means the
two spellings cannot drift apart.

HConfigDriverRules regains negate_with, negation_default_when, and
negation_sub as declared fields so type checkers accept them; an after-model
validator folds their contents into the unified negation list in the order
DEFAULT, REPLACE, REGEX_SUB, matching load_driver_rules() and reproducing the
v3 resolution priority. They are constructor inputs only.

Prove parity with the real thing: tests/integration/v3_scenarios.py mirrors
golden config's _get_hierconfig_remediation and imports under both majors.
test_v3_baseline.py compares 28 scenarios against a committed v3.7.0
recording on every push; test_v3_differential.py (marker v3_differential,
deselected by default) diffs against a live v3 install. All 28 scenarios
match v3 byte for byte.

Out of scope, and documented as such: depth() became a property, several call
sites raise typed exceptions instead of ValueError, and the EOS/NX-OS/XR
views return data where v3 raised NotImplementedError.

Claude-Session: https://claude.ai/code/session_01HbMucj2WqnL685BRUcFhZT
Code review found that v3's own custom-driver documentation teaches
driver.rules.negate_with.append(...) in three places (docs/user/custom-drivers.md
lines 193, 276, 410 at tag v3.7.0). Folding those fields into the unified
negation list once at validation meant that idiom still ran, silently did
nothing, and produced a different -- potentially destructive -- remediation.
The premise behind the constructor-only decision was wrong.

Replace the after-model validator with HConfigDriverRules.all_negation_rules(),
which resolves `negation` plus the three v3 fields on every lookup.
driver.negate_with() and HConfigChild.negate() read through it. Appending to
either spelling now takes effect, and because nothing is mutated, re-validating
a rules model no longer duplicates every v3 rule -- HConfigDriverRules(
**base.model_dump(), negate_with=[...]) was silently doubling them. The no-v3-
rules path returns `negation` itself, so the common case allocates nothing.

Add options:negation_appended to the shared scenario module, exercising the
append idiom end to end. It matches v3.7.0 byte for byte, as do the other 28.

Also from the review:
- Restore HConfigChild.use_default_for_negation(), the one v3 public name still
  missing. The compatibility page claimed every v3 name worked; now it does.
- Correct CHANGELOG.md, which listed names under Removed that this branch
  retains, and add the missing issue references.
- Fix the hconfig_v2_os_v3_platform_mapper doctest: Platform is a str-mixin
  enum, so the repr is quoted.

Claude-Session: https://claude.ai/code/session_01HbMucj2WqnL685BRUcFhZT
@jtdub
jtdub requested a review from aedwardstx as a code owner August 27, 2026 21:19
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown

Review in progress

  • Gather context (PR body, changed files list)
  • Run git diff next...HEAD and read changed files
  • Run the hier-config-review skill checklist (models/typing, tests/TDD/coverage, changelog, docs, driver patterns)
  • Post findings grouped by severity + verdict against AGENTS.md checklist

View branch

@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown

Claude finished @jtdub's task in 3m 42s —— View job


Review complete ✅

  • Gather context (PR body, changed files list)
  • Run git diff origin/next...HEAD and read changed files
  • Run the hier-config-review skill checklist (models/typing, tests/TDD/coverage, changelog, docs, driver patterns)
  • Post findings grouped by severity + verdict against AGENTS.md checklist

Gates: ./scripts/build.py lint → clean. ./scripts/build.py pytest --coverage → 846 passed, 97.47% coverage. mkdocs build --strict → clean.


Blockers

None.

Should fix

  • docs/user/migrating-from-v3.md:166-189 — the "Behavior changes to review" intro claims "These are the changes that the v3 compatibility surface cannot absorb", but the list underneath includes three items that were never candidates for name-alias coverage in the first place: future() negation resolution, structured input (XML/JSON) rejection, and custom driver wiring — all pre-existing v4 behavior changes unrelated to renamed/removed names. Framing them alongside the genuine compatibility gaps (depth property, typed exceptions, completed views) overstates what the v3-compat surface was ever meant to cover and could read as if it almost handled them. Consider splitting "genuine v3-compat gaps" from "other v4 behavior changes worth reviewing" so the claim stays accurate.
  • docs/user/v3-compatibility.md:122-137 ("One limit") lists four items (adds XML/JSON rejection to depth/exceptions/views), while the PR description says "Three v4 changes cannot be absorbed by a name alias". Not a doc bug per se (the PR body isn't shipped), but worth reconciling so the next person copying that framing into release notes doesn't drop the fourth item.

Nits

  • tests/unit/test_v3_compat.py:237-252 (test_load_hconfig_v2_options_from_file) takes tags_file_path as a fixture parameter and ends with assert Path(tags_file_path).exists() — unrelated to what the test name/body is exercising (load_hconfig_v2_options_from_file). Looks like a copy-paste leftover from a neighboring tags test; the unrelated fixture arg and assertion can be dropped without weakening coverage.

Notes (not findings, for context)

  • all_negation_rules() (hier_config/platforms/driver_base.py:168) and both call sites (driver_base.py:237, child.py:275,305) are consistent — no stale self.rules.negation reads left over from the earlier constructor-only design.
  • v3-compat models (NegationDefaultWithRule/NegationDefaultWhenRule/NegationSubRule) correctly subclass the local BaseModel, use tuple[MatchRule, ...], and delegate via to_negation_rule() — no second implementation.
  • dict[str, Any] in utils.py's v3 wrappers matches the existing convention already used by load_driver_rules()/load_tag_rules() in the same file (untyped YAML/JSON ingestion) — not a new "no Any" violation.
  • poetry.lock/pyproject.toml diff is a routine relock; no packages added or removed, so the "no new runtime dependencies" rule holds.
  • The dual-version scenario module (tests/integration/v3_scenarios.py) and its baseline/differential tests are genuinely thorough: 29 scenarios recorded against v3.7.0, all passing in the normal suite, plus an opt-in live-v3 differential check.
  • Commit messages are well-formed (imperative mood, ≤72-char subjects, bodies explain why); the fix-up commit 2086635 is a good example of taking review feedback seriously (fixed a real correctness bug — the negation-fields-folded-once-at-validation issue — before merging).

Verdict against AGENTS.md "Before Opening a PR"

  • poetry run ./scripts/build.py lint-and-test exits 0
  • Tests written first (TDD) and cover the change — extensive tests/unit/test_v3_compat.py plus the v3/v4 parity suite
  • CHANGELOG.md updated under ## [Unreleased], including a correction to a previously-wrong ### Removed section
  • Docs updated (docs/user/v3-compatibility.md added and in nav, docs/user/migrating-from-v3.md reframed, docs/dev/testing.md updated); mkdocs build --strict passes
  • Commit messages follow CONTRIBUTING.md style

Pass. No blockers found; the two "should fix" items are doc-wording accuracy issues, not code or hard-rule violations.

PR review found that "Behavior changes to review" in the migration guide
claimed every item under it was something the v3 compatibility surface could
not absorb. Three of the six -- future() negation resolution, structured input
rejection, and custom driver wiring -- were never candidates for name-alias
coverage. Grouping them with the genuine gaps overstated what the surface was
meant to cover.

Split the section. "Gaps the v3 compatibility surface cannot cover" holds the
four changes that reach code keeping the v3 spellings: the depth property, the
typed exceptions, the completed views, and XML/JSON rejection -- which belongs
there because the restored get_hconfig() raises where v3 did not. "Other v4
changes worth reviewing" holds the rest.

Both pages now state four consistently; the compatibility page said four while
the PR description said three.

Also drop a copy-paste leftover in test_load_hconfig_v2_options_from_file: it
took the tags_file_path fixture and asserted the file exists, neither of which
relates to what it exercises. Its remaining assertion now reads through
all_negation_rules() rather than the raw negation list.

Claude-Session: https://claude.ai/code/session_01HbMucj2WqnL685BRUcFhZT
@claude

claude Bot commented Aug 27, 2026

Copy link
Copy Markdown

Claude finished @jtdub's task in 1m 30s —— View job


Review complete ✅

  • Gather context (PR body, changed files list)
  • Run git diff origin/next...HEAD and read changed files (branch updated since last review: 389a3d2 increment version, 5468c0d split v3-compat gaps from unrelated v4 changes)
  • Run the hier-config-review skill checklist (models/typing, tests/TDD/coverage, changelog, docs, driver patterns)
  • Post findings grouped by severity + verdict against AGENTS.md checklist

Gates: ./scripts/build.py lint → clean. ./scripts/build.py pytest --coverage → 846 passed, 97.47% coverage. mkdocs build --strict → clean.


Both "Should fix" items from the previous review pass are resolved in 5468c0d:

  • docs/user/migrating-from-v3.md:166-193 now splits "Gaps the v3 compatibility surface cannot cover" (depth property, typed exceptions, completed views, structured-input rejection) from "Other v4 changes worth reviewing" (future() negation resolution, custom driver wiring) — the section no longer claims the latter three were candidates for name-alias coverage.
  • docs/user/v3-compatibility.md:122-137 ("The limit") now lists the same four items as the migration guide (added structured-input rejection), so the two pages and the PR description are consistent.
  • tests/unit/test_v3_compat.py:237-251 (test_load_hconfig_v2_options_from_file) dropped the unrelated tags_file_path fixture/assertion and now checks driver.rules.all_negation_rules() instead of the raw negation list — matches what the test name says it exercises.

Blockers

None.

Should fix

None.

Nits

None.

Verdict against AGENTS.md "Before Opening a PR"

  • poetry run ./scripts/build.py lint-and-test exits 0
  • Tests written first (TDD) and cover the change — tests/unit/test_v3_compat.py plus the 29-scenario v3/v4 parity suite
  • CHANGELOG.md updated under ## [Unreleased], including the correction to the ### Removed section entries now noting "the v3 name is retained"
  • Docs updated (docs/user/v3-compatibility.md added and in nav, docs/user/migrating-from-v3.md reframed) and internally consistent; mkdocs build --strict passes
  • Commit messages follow CONTRIBUTING.md style — 5468c0d's body explains why clearly, citing the prior review finding

Pass. No blockers, no should-fix items, no nits remaining.
·

@jtdub
jtdub merged commit d03fbac into next Aug 27, 2026
7 checks passed
@jtdub
jtdub deleted the restore-v3-names-in-v4 branch August 27, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant